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Why GAO Did This Study 

The screening of airport passengers 
and their checked baggage is a 
critical component in securing our 
nation's commercial aviation 
system. Since May 2003, GAO has 
issued six products related to 
screener training and performance. 
This report updates the information 
presented in the prior products and 
incorporates results from GAO's 
survey of 155 Federal Security 
Directors — the ranking 
Transportation Security 
Administration (TSA) authority 
responsible for the leadership and 
coordination of TSA security 
activities at the nation's 
commercial airports. Specifically, 
this report addresses (1) actions 
TSA has taken to enhance training 
for passenger and checked baggage 
screeners and screening 
supervisors, (2) how TSA ensures 
that screeners complete required 
training, and (3) actions TSA has 
taken to measure and enhance 
screener performance in detecting 
threat objects. 



What GAO Recommends 



GAO is recommending that the 
Secretary of Homeland Security 
direct TSA to develop a plan for 
completing the deployment of high- 
speed connectivity at airport 
training facilities, and establish and 
communicate appropriate internal 
controls for monitoring the 
completion of training. 

TSA reviewed a draft of this report 
and generally agreed with GAO's 
findings and recommendations. 



www.gao.gov/cgi-bin/getrpt7GAO-05-457. 

To view the full product, including the scope 
and methodology, click on the link above. 
For more information, contact Cathleen 
Berrick, 202-512-8777, berrickc@gao.gov. 



What GAO Found 

TSA has initiated a number of actions designed to enhance screener training, 
such as updating the basic screener training course. TSA also established a 
recurrent training requirement and introduced the Online Learning Center, 
which makes self-guided training courses available over TSA's intranet and 
the Internet. Even with these efforts, Federal Security Directors reported 
that insufficient screener staffing and a lack of high-speed Internet/intranet 
connectivity at some training facilities have made it difficult to fully utilize 
training programs and to meet the recurrent training requirement of 3 hours 
per week, averaged over a quarter year, within regular duty hours. TSA 
acknowledged that challenges exist in recurrent training delivery and is 
taking steps to address these challenges, including factoring training into 
workforce planning efforts and distributing training through written 
materials and CD-ROMs. However, TSA has not established a plan 
prioritizing the deployment of high-speed Internet/intranet connectivity to all 
airport training facilities to facilitate screener access to training materials. 

TSA lacks adequate internal controls to provide reasonable assurance that 
screeners receive legislatively mandated basic and remedial training, and to 
monitor its recurrent training program. Specifically, TSA policy does not 
clearly specify the responsibility for ensuring that screeners have completed 
all required training. In addition, TSA officials have no formal policies or 
methods for monitoring the completion of required training and were unable 
to provide documentation identifying the completion of remedial training. 

TSA has implemented and strengthened efforts to measure and enhance 
screener performance. For example, TSA has increased the number of 
covert tests it conducts at airports, which test screeners' ability to detect 
threat objects on passengers, in their carry-on baggage, and in checked 
baggage. These tests identified that overall, weaknesses and vulnerabilities 
continue to exist in passenger and checked baggage screening systems at 
airports of all sizes, at airports with federal screeners, and at airports with 
private-sector screeners. While these test results are an indicator of 
performance, they cannot solely be used as a comprehensive measure of any 
airport's screening performance or any individual screener's performance. 
We also found that TSA's efforts to measure and enhance screener 
performance have primarily focused on passenger screening, not checked 
baggage screening. For example, TSA only uses threat image software on 
passenger screening X-ray machines, and the recertification testing program 
does not include an image recognition module for checked baggage 
screeners. TSA is taking steps to address the overall imbalance in passenger 
and checked baggage screening performance data. TSA also established 
performance indexes for the passenger and checked baggage screening 
systems, to identify an overall desired level of performance. However, TSA 
has not established performance targets for each of the component 
indicators that make up the performance indexes, including performance 
targets for covert testing. TSA plans to finalize these targets by the end of 
fiscal year 2005. 
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A 

^ G A O 

Accountability * Integrity * Reliability 

United States Government Accountability Office 
Washington, DC 20548 



May 2, 2005 

The Honorable John Mica 
Chairman, Subcommittee on Aviation 
Committee on Transportation and Infrastructure 
House of Representatives 

Dear Mr. Chairman: 

The screening of airport passengers and their checked baggage is a critical 
component in securing our nation's commercial aviation system. In an 
effort to strengthen the security of commercial aviation, the President 
signed the Aviation and Transportation Security Act on 
November 19, 2001. The act created the Transportation Security 
Administration (TSA) and mandated actions designed to strengthen 
aviation security, including requiring that TSA assume responsibility for 
conducting passenger and checked baggage screening at over 450 
commercial airports in the United States by November 19, 2002. It has 
been over 2 years since TSA assumed this responsibility, and the agency 
has spent billions of dollars and implemented a wide range of initiatives to 
enhance its passenger and checked baggage screening operations. Despite 
the attention to passenger and checked baggage screening operations, 
however, concerns about the effectiveness of the screening system 
remain. For example, covert testing conducted by TSA's Office of Internal 
Affairs and Program Review and the Department of Homeland Security's 
(DHS) Office of Inspector General identified weaknesses in the ability of 
screeners to detect threat objects. (The results of our analysis of TSA's 
covert testing data and test program are included in a separate classified 
GAO report.) 

To determine the progress TSA has made in strengthening its passenger 
and checked baggage screening operations, the Subcommittee on 
Aviation, House Committee on Transportation and Infrastructure, 
requested that we examine TSA efforts to train screeners and to measure 
and enhance screener performance. Since we began our work in May 2003, 
we have issued six products that address issues related to screener 
training and performance, including four to this Subcommittee (see app. 
I). This report updates some of the information presented in our prior 
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products. In addition, it incorporates results from our surveys of 155 
Federal Security Directors (FSD). 1 The surveys were designed to obtain 
information related to, among other issues, TSA's efforts to train screeners 
and supervisors and assess screener performance in detecting threat 
objects. Specifically, this report addresses the following questions: 
(1) What actions has TSA taken to enhance training for passenger and 
checked baggage screeners and screening supervisors? (2) How does TSA 
ensure that screeners complete required training? (3) What actions has 
TSA taken to measure and enhance screener performance in detecting 
threat objects? 

In conducting our work, we reviewed TSA documentation related to 
screener training requirements and performance testing. We also analyzed 
data from our survey responses from all 155 FSDs about screener training, 
supervision and performance. We also visited 29 airports of various sizes 
and geographic locations to obtain a cross-section of all airports, including 
the 5 airports with private-sector screeners. 2 To gain a better 
understanding of training and performance issues, during these visits, we 
interviewed FSDs, members of their management teams, passenger and 
checked baggage screeners, and airport officials. However, information 
obtained during these visits cannot be generalized to all airports across the 
nation. Additionally, we interviewed officials at TSA headquarters and 
TSA's transportation security laboratory about their experiences with 
training and screener performance. We compared TSA practices and 



lr The Federal Security Director is the ranking TSA authority responsible for the leadership 
and coordination of TSA security activities at the nation's commercial airports. We sent 
two surveys — a general survey and an airport-specific survey — to all 155 Federal Security 
Directors on March 23, 2004. In the general survey, we asked each Federal Security 
Director to answer security-related questions that pertain to all of the airports for which 
he/she is responsible. In the airport-specific survey, we asked Federal Security Directors a 
number of airport-specific questions about screening and other security concerns for one 
or two airports, depending on the number of airports they were responsible for. By early 
May 2004, we had received responses from 100 percent of the Federal Security Directors 
for both surveys. 

2 Pursuant to section 108 of the Aviation and Transportation Security Act (ATSA), TSA 
conducted a 2-year private screening pilot program at five airports — one in each airport 
security category (Pub. L. No. 107-71). The mission of the pilot, as defined by TSA, was to 
test the effectiveness of using private screening contractors in a post-September 11 
environment. The pilot concluded on November 18, 2004. On November 19, 2004, 
consistent with ATSA, TSA began allowing airports to apply to opt out of using federal 
screeners in favor of private contractors. For additional information on TSA's progress in 
developing the opt-out program, see GAO, Aviation Security: Preliminary Observations 
on TSA's Progress to Allow Airports to Use Private Passenger and Baggage Screening 
Services, GAO-05-126 (Washington, D.C.: Nov. 19, 2004). 
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procedures for monitoring completion of training with the Comptroller 
General's Standards for Internal Control in the Federal Government. We 
assessed the reliability of the data we acquired from TSA regarding 
screener testing and training completion, and found the data to be 
sufficiently reliable for our purposes. A more detailed description of our 
scope and methodology is contained in appendix II. 

We conducted our work from May 2003 through April 2005 in accordance 
with generally accepted government auditing standards. 



RGSllltS in Brief TSA has initiated a number of actions designed to enhance passenger 

screener, checked baggage screener, and screener supervisor training. 
However, at some airports screeners encountered difficulty accessing and 
completing recurrent (refresher) training because of technological and 
staffing constraints. Among the actions TSA has taken to enhance training 
are changes and updates to the basic training program. For example, TSA 
added a modular dual-function course during basic training that covers 
passenger and checked baggage screening functions and allows newly 
hired screeners to perform either function upon completion of the 
training. TSA also established a requirement for recurrent screener 
training and developed and introduced the Online Learning Center, which 
makes self-guided training courses available to employees over TSA's 
intranet and the Internet. In addition, TSA has provided Federal Security 
Directors with hands-on training tools to use for local recurrent training 
and testing. Finally, TSA has taken steps to provide leadership and 
technical training to Screening Supervisors. Despite these improvements, 
some Federal Security Directors, in response to open-ended survey 
questions, identified a desire for more training in specific areas, including 
leadership, communication, and supervision. Further in survey responses, 
Federal Security Directors reported that largely because of insufficient 
screener staffing, screeners were not always able to meet the recurrent 
training requirement within regular duty hours. Federal Security Directors 
at some airports also reported ongoing problems with a lack of high-speed 
Internet/intranet connectivity, which severely limited screener access to 
the Online Learning Center and diminished its value as a learning tool. As 
of October 2004, nearly half of the screener workforce did not have high- 
speed access to the Online Learning Center at their training facility. TSA 
has acknowledged that challenges exist in recurrent screener training 
delivery and is taking steps to address these challenges, including 
factoring training requirements into workforce planning efforts and 
distributing training through written materials and CD-ROMs until full 
Internet/intranet connectivity is achieved. However, TSA does not have a 
plan for prioritizing and scheduling the deployment of high-speed 
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connectivity to all airport training facilities once funding is available. The 
absence of such a plan limits TSA's ability to make prudent decisions 
about how to move forward with deploying connectivity to all airports to 
provide screeners access to online training. 

TSA lacks adequate internal controls to provide reasonable assurance that 
screeners are receiving legislatively mandated basic and remedial training, 
and to monitor the status of its recurrent training program. The 
Comptroller General's Standards for Internal Control call for (1) areas of 
authority and responsibility to be clearly defined and appropriate lines of 
reporting established, (2) transactions and other significant events to be 
documented clearly and documentation to be readily available for 
examination, and (3) controls generally to be designed to ensure that 
ongoing monitoring occurs in the course of normal operations. 3 In 
addition, the standards also require that information be communicated 
within an organization to enable individuals to carry out their internal 
control responsibilities. However, our review of TSA's training program 
noted weaknesses in each of these control areas. First, TSA policy does 
not clearly define the responsibility for ensuring that screeners have 
completed all required training. Second, TSA officials were unable to 
produce documentation of remedial training completion for our 
examination. Third, TSA has no formal policies for monitoring the 
completion of required training. Moreover, because of the lack of high- 
speed Internet/intranet connectivity at some airports, staff had to manually 
input training data that would otherwise be recorded automatically by the 
Online Learning Center, making it challenging for some airports to keep 
accurate and up-to-date training records. TSA headquarters officials 
acknowledged that it can be difficult for airports to keep the Online 
Learning Center up-to-date with the most recent training records without 
high-speed connectivity. 

TSA has improved its efforts to measure and enhance screener 
performance. In September 2003, we first reported on the need for TSA to 
strengthen its efforts to measure and enhance screener performance. At 
that time, TSA had collected limited data on screener performance. 
Specifically, limited covert testing — unannounced undercover tests in 
which TSA agents attempt to pass threat objects through screening 
checkpoints and in checked baggage — had been performed, the Threat 



3 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-2 1.3.1 
(Washington, D.C.: November 1999). 
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Image Projection system was not fully operational, 4 and TSA had not fully 
implemented the annual screener recertification program. 5 Since then, TSA 
has implemented and strengthened efforts to collect screener performance 
data as part of its overall effort to enhance screener performance. For 
example, TSA has increased the number of covert tests it conducts at 
airports. These tests have identified that overall, weaknesses and 
vulnerabilities continue to exist in the passenger and checked baggage 
screening systems at airports of all sizes, at airports with federal 
screeners, and at airports with private-sector screeners. While these test 
results are an indicator of screener performance, they cannot solely be 
used as a comprehensive measure of any airport's screening performance 
or any individual screener's performance. TSA has also implemented other 
efforts to measure and enhance screener performance. However, these 
efforts have primarily focused on passenger screening, not checked 
baggage screening. Specifically, the Threat Image Projection system is 
only available for passenger screening; the recertification testing program 
does not include an image recognition module for checked baggage 
screeners; and the screener performance improvement study focused 
solely on passenger screeners. TSA is taking steps to address the overall 
imbalance in passenger and baggage screening performance data, 
including working toward implementing the Threat Image Projection 
system for checked baggage screening and developing an image 
recognition module for checked baggage screener recertification. TSA has 
also implemented a number of other improvements, which it identified in a 
passenger screener performance improvement study and incorporated 
into a screener performance improvement plan. Furthermore, TSA has 
established two performance indexes for the screening systems — one for 
passenger and one for checked baggage screening. 6 These indexes 
measure overall performance through a composite of component 
indicators. However, TSA has not established performance targets for 
each of the component indicators — such as covert testing — which would 



4 The Threat Image Projection system is designed to test screeners' detection capabilities by 
projecting threat images, including images of guns and explosives, into bags as they are 
screened. Screeners are responsible for positively identifying the threat image and calling 
for the bag to be searched. 

5 ATSA requires that TSA collect performance information on all passengers and baggage 
screeners by conducting an annual proficiency evaluation to ensure each screener 
continues to meet all qualifications and standards related to the functions he or she 
performs. 

6 TSA's performance indexes show how well screening systems are functioning on a scale of 
1 to 5, using a weighted average of the values of four distinct performance indicators. 
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allow it to draw more meaningful conclusions about its performance and 
most effectively direct its improvement efforts. Although TSA has not yet 
established performance targets for each of the component indicators, 
TSA plans to finalize performance targets for the indicators by the end of 
fiscal year 2005. 

Certain information we obtained and analyzed regarding screener training 
and performance is classified or is considered by TSA to be sensitive 
security information. Accordingly, the results of our review of this 
information are not included in this report. 7 

To help ensure access to and completion of required training, we are 
making recommendations to the Secretary of the Department of Homeland 
Security to direct the Assistant Secretary, Transportation Security 
Administration, to (1) develop a plan for completing the deployment of 
high-speed Internet/intranet connectivity to all TSA's airport training 
facilities and (2) establish appropriate responsibilities and other internal 
controls for monitoring and documenting screener compliance with 
training requirements. 

We provided a draft copy of this report to DHS for review. DHS, in its 
written comments, generally concurred with the findings and 
recommendations in the report and agreed that efforts to implement our 
recommendations are critical to successful passenger and checked 
baggage screening training and performance. DHS described some actions 
TSA had taken or planned to take to implement these recommendations. 
DHS also stated that TSA had already developed a plan for prioritizing and 
scheduling the deployment of high-speed Internet/intranet connectivity to 
all TSA airport training facilities. However, although we requested a copy 
of the plan several times during our review and after receiving written 
comments from DHS, TSA did not provide us with a copy of this plan. 
Therefore, we cannot assess the extent to which the plan DHS referenced 
in its written comments fulfills our recommendation. Additionally, DHS 
stated that it is taking steps to define responsibilities for monitoring the 



7 We issued two additional reports detailing the results of our review, which discuss 
information deemed to be classified or sensitive security information. The report 
containing sensitive security information is GAO, Aviation Security: Screener Training 
and Performance Measurement Strengthened, but More Work Remains, GAO-05-143SU 
(Washington D.C.: Feb. 28, 2005). The report containing classified and sensitive security 
information is GAO, Aviation Security: Results of Transportation Security 
Administration's Covert Testing for Passenger and Checked Baggage Screening for 
September 2002 through September 2004, GAO-05-437C (Washington D.C.: Apr. 7, 2005). 
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completion of required training, and to insert this accountability into 
performance plans of all TSA supervisors. The full text of DHS's comments 
is included in appendix V. 

Background ^ e P er f ormance °f passenger and checked baggage screeners in 

& detecting threat objects at the nation's airports has been a long-standing 

concern. In 1978, screeners failed to detect 13 percent of the potentially 
dangerous objects that Federal Aviation Administration (FAA) agents 
carried through airport screening checkpoints during tests. In 1987, 
screeners did not detect 20 percent of the objects in similar tests. In tests 
conducted during the late 1990s, as the testing objects became more 
realistic, screeners' abilities to detect dangerous objects declined further. 
In April 2004, we, along with the DHS Office of the Inspector General 
(OIG), testified that the performance of screeners continued to be a 
concern. More recent tests conducted by TSA's Office of Internal Affairs 
and Program Review (OIAPR) also identified weaknesses in the ability of 
screeners to detect threat objects, and separate DHS OIG tests identified 
comparable screener performance weaknesses. In its July 2004 report, The 
National Commission on Terrorist Attacks Upon the United States, known 
widely as the 9/11 Commission, also identified the need to improve 
screener performance and to better understand the reasons for 
performance problems. 8 

After the terrorist attacks of September 11, 2001, the President signed the 
Aviation and Transportation Security Act (ATSA) into law on November 
19, 2001, with the primary goal of strengthening the security of the nation's 
aviation system. ATSA created TSA as an agency with responsibility for 
securing all modes of transportation, including aviation. 9 As part of this 
responsibility, TSA oversees security operations at the nation's more than 
450 commercial airports, including passenger and checked baggage 
screening operations. Prior to the passage of ATSA, air carriers were 
responsible for screening passengers and checked baggage, and most used 
private security firms to perform this function. FAA was responsible for 
ensuring compliance with screening regulations. 



The National Commission on Terrorist Attacks Upon the United States: The 9/11 
Commission Report (Washington, D.C., July 2004). 

Consistent with ATSA, TSA was created as an agency within the Department of 
Transportation (DOT) with responsibility for securing all modes of transportation, 
including aviation. The Homeland Security Act of 2002, signed into law on 
November 25, 2002, transferred TSA from the DOT to the new Department of Homeland 
Security (Pub. L. No. 107-296). 
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Today, TSA security activities at airports are overseen by FSDs. Each FSD 
is responsible for overseeing security activities, including passenger and 
checked baggage screening, at one or more commercial airports. TSA 
classifies the over 450 commercial airports in the United States into one of 
five security risk categories (X, I, II, III, and IV) based on various factors, 
such as the total number of takeoffs and landings annually, the extent to 
which passengers are screened at the airport, and other special security 
considerations. In general, category X airports have the largest number of 
passenger boardings and category IV airports have the smallest. TSA 
periodically reviews airports in each category and, if appropriate, updates 
airport categorizations to reflect current operations. Figure 1 shows the 
number of commercial airports by airport security category as of 
December 2003. 

Figure 1 : Commercial Airports by Airport Security Category as of December 2003 

21 Airports 



61 Airports 



43.3% 



197 Airports 



Category 
X 
I 




51 Airports 



124 Airports 



Total: 454 



IV 



Source: GAO analysis of TSA data. 



Note: TSA periodically reviews and updates airport categories to reflect current operations. We used 
the categories in place in December 2003 to conduct our analyses during this review. 
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In addition to establishing TSA and giving it responsibility for passenger 
and checked baggage screening operations, ATSA set forth specific 
enhancements to screening operations for TSA to implement, with 
deadlines for completing many of them. These requirements included 

• assuming responsibility for screeners and screening operations at more 
than 450 commercial airports by November 19, 2002; 

• establishing a basic screener training program composed of a minimum of 
40 hours of classroom instruction and 60 hours of on-the-job training; 

• conducting an annual proficiency review of all screeners; 

• conducting operational testing of screeners; 10 

• requiring remedial training for any screener who fails an operational test; 
and 

• screening all checked baggage for explosives using explosives detection 
systems by December 31, 2002. 11 

Passenger screening is a process by which authorized TSA personnel 
inspect individuals and property to deter and prevent the carriage of any 
unauthorized explosive, incendiary, weapon, or other dangerous item 
aboard an aircraft or into a sterile area. 12 Passenger screeners must inspect 
individuals for prohibited items at designated screening locations. 13 The 
four passenger screening functions are: 



10 TSA defines an operational screening test as any covert test of a screener conducted by 
TSA, on any screening function, to assess the screener's threat item detection ability and/or 
adherence to TSA-approved procedures. 

n Pursuant to the Homeland Security Act, the deadline for screening all checked baggage 
using explosive detection systems was, in effect, extended until December 31, 2003. 

12 Sterile areas are located within the terminal where passengers wait after screening to 
board departing aircraft. Access to these areas is controlled by TSA screeners at 
checkpoints where they conduct physical screening of passengers and their carry-on 
baggage for weapons and explosives. 

13 Screeners must deny passage beyond the screening location to any individual or property 
that has not been screened or inspected in accordance with passenger screening standard 
operating procedures. If an individual refuses to permit inspection of any item, that item 
must not be allowed into the sterile area or aboard an aircraft. 
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• X-ray screening of property, 

• walk-through metal detector screening of individuals, 

• hand- wand or pat-down screening of individuals, and 

• physical search of property and trace detection for explosives. 

Checked baggage screening is a process by which authorized security 
screening personnel inspect checked baggage to deter, detect, and prevent 
the carriage of any unauthorized explosive, incendiary, or weapon 
onboard an aircraft. Checked baggage screening is accomplished through 
the use of explosive detection systems 14 (EDS) or explosive trace detection 
(ETD) systems, 15 and through the use of alternative means, such as manual 
searches, K-9 teams, and positive passenger bag match, 16 when EDS and 
ETD systems are unavailable on a temporary basis. Figure 2 provides an 
illustration of passenger and checked baggage screening operations. 



Explosive detection systems use probing radiation to examine objects inside baggage and 
identify the characteristic signatures of threat explosives. EDS equipment operates in an 
automated mode. 

15 Explosive trace detection works by detecting vapors and residues of explosives. Human 
operators collect samples by rubbing bags with swabs, which are chemically analyzed to 
identify any traces of explosive materials. 

16 Positive passenger bag match is an alternative method of screening checked baggage, 
which requires that the passenger be on the same aircraft as the checked baggage. 
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Figure 2: Passenger Checkpoint and Checked Baggage Screening Operations 
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| Passenger and baggage screening functions 
Source: GAO and Nova Development Corporation. 
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There are several positions within TSA for employees that perform and 
directly supervise passenger and checked baggage screening functions. 
Figure 3 provides a description of these positions. 



Figure 3: Description of Screening-Related Positions 



Positions 

(Number of positions TSA 
was budgeted for in FY 04) 



Major functions 



Screening 
Manager 

(1,210 positions) 



(1) manages screening operations; (2) ensures quality and consistency of screening procedures; (3) schedules screening 
personnel to screening operations; (4) manages overall screening workforce issues; (5) manages external relationships; 
and (5) interprets technical aspects of TSA policies, regulations, and directives. 



Screening 
Supervisor 

(3,419 positions) 




(1) identifies, distributes, and balances workload and tasks among screeners; (2) makes necessary 
adjustments to accomplish the screening workload; (3) trains or arranges for the technical training of 
screeners; (4) monitors and reports on the status and progress of screening work; (5) maintains records of 
work accomplishments and administrative information; (6) represents a team of screeners for the 
purpose of obtaining resources, and secures needed information or decisions from the supervisor on 
major work problems and issues that arise; (7) serves as a coach and facilitator to a team of 
screeners; (8) resolves simple, informal complaints of screeners and refers formal grievances to 
the appropriate management official; and (9) approves leave as delegated by management. 



Lead 
Screener 

(3,891 positions) 



(1) provides frontline security protection to air travelers, airports, and airplanes; 
(2) wands and performs pat-down searches, operates an X-ray machine, 
screens baggage, and reviews tickets; (3) identifies dangerous objects on 
passengers or in baggage or cargo, and prevents these objects from 
being transported onto aircraft; and (4) uses diverse electronic 
detection and imaging equipment. 



Screener 

(37,023 positions) 



(1) performs preboard screening of persons and their carry-on 
or checked baggage; (2) implements security-screening 
procedures that are central to TSA objectives and will 
serve to protect the traveling public by preventing any 
deadly or dangerous objects from being transported 
onto the aircraft; and (3) assists in monitoring the 
flow of passengers through the screening 
checkpoint. 



Source: GAO analysis of TSA data. 
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To prepare screeners to perform screening functions, to keep their skills 
current, and to address performance deficiencies, TSA provides three 
categories of required screener training. Table 1 provides a description of 
the required training. 



Table 1 : Categories of Required Training Provided to TSA Screeners 



Category of 

training Description of training requirement 

Basic training ATSA requires newly hired screeners to complete basic training 

composed of a minimum of 40 hours of classroom instruction and 
60 hours of on-the-job training before they independently perform 
screening functions. 

Recurrent training TSA policy requires screeners to complete 3 hours of skills 
refresher training per week averaged over each quarter. 

Remedial training ATSA requires screeners who fail any operational test to complete 
remedial training on the function they failed before they resume 
performing that function. 

Source: ATSA and TSA. 



In September 2003, we reported on our preliminary observations of TSA's 
efforts to ensure that screeners were effectively trained and supervised 
and to measure screener performance. 17 We found that TSA had 
established and deployed a basic screener training program and required 
remedial training but had not fully developed or deployed a recurrent 
training program for screeners or supervisors. We also reported that TSA 
had collected limited data to measure screener performance. Specifically, 
TSA had conducted limited covert testing, the Threat Image Projection 
System was not fully operational, and TSA had not implemented the 
annual screener proficiency testing required by ATSA. In subsequent 
products, we reported progress TSA had made in these areas and 
challenges TSA continued to face in making training available to screeners 
and in measuring and enhancing screener performance. A summary of our 
specific findings is included in appendix I. 



17 GAO, Airport Passenger Screening: Preliminary Observations on Progress Made and 
Challenges Remaining, GAO-03-1173 (Washington, D.C.: Sept. 24, 2003). 
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TSA Has Enhanced 
and Expanded 
Training, but Some 
Screeners Have 
Encountered 
Difficulty Accessing 
and Completing 
Recurrent Training 



TSA has taken a number of actions to enhance the training of screeners 
and Screening Supervisors but has encountered difficulties in providing 
access to recurrent training. TSA has enhanced basic training by, among 
other things, adding a dual-function (passenger and checked baggage) 
screening course for new employees. Furthermore, in response to the 
need for frequent and ongoing training, TSA has implemented an Online 
Learning Center with self-guided training courses available to employees 
over TSA's intranet and the Internet and developed and deployed a 
number of hands-on training tools. 18 Moreover, TSA now requires 
screeners to participate in 3 hours of recurrent training per week, 
averaged over each quarter year. TSA has also implemented leadership 
and technical training programs for Screening Supervisors. However, 
some FSDs, in response to open-ended survey questions, identified a 
desire for more training in specific areas, including leadership, 
communication, and supervision. Further, despite the progress TSA has 
made in enhancing and expanding screener and supervisory training, TSA 
has faced challenges in providing access to recurrent training. FSDs 
reported that insufficient staffing and a lack of high-speed 
Internet/intranet connectivity at some training facilities have made it 
difficult to fully utilize these programs and to meet training requirements. 
TSA has acknowledged that challenges exist in recurrent screener training 
delivery and is taking steps to address these challenges, including 
factoring training requirements into workforce planning efforts and 
distributing training through written materials and CD-ROMs until full 
Internet/intranet connectivity is achieved. However, TSA does not have a 
plan for prioritizing and scheduling the deployment of high-speed 
connectivity to all airport training facilities once funding is available. The 
absence of such a plan limits TSA's ability to make prudent decisions 
about how to move forward with deploying connectivity to all airports to 
provide screeners access to online training. 



TSA Has Enhanced Basic TSA has enhanced its basic screener training program by updating the 
Screener Training training to reflect changes to standard operating procedures, deploying a 

new dual-function (passenger and checked baggage screening) basic 
training curriculum, and allowing the option of training delivery by local 
staff. As required by ATSA, TSA established a basic training program for 



18 The Online Learning Center is TSA's central, official electronic source of all training and 
related performance/development accomplishments for TSA employees. It serves as the 
delivery platform for online training and is the official repository for TSA training records. 
TSA launched the Online Learning Center on October 31, 2003. 
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screeners composed of a minimum of 40 hours of classroom instruction 
and 60 hours of on-the-job training. TSA also updated the initial basic 
screener training courses at the end of 2003 to incorporate changes to 
standard operating procedures and directives, which contain detailed 
information on how to perform TSA-approved screening methods. 
However, a recent study by the DHS OIG found that while incorporating 
the standard operating procedures into the curricula was a positive step, a 
number of screener job tasks were incompletely addressed in or were 
absent from the basic training courses. 19 

In addition to updates to the training curriculum, in April 2004, TSA 
developed and implemented a new basic screener training program, dual- 
function screener training that covers the technical aspects of both 
passenger and checked baggage screening. Initially, new hire basic 
training was performed by a contractor and provided a screener with 
training in either passenger or checked baggage screening functions. A 
screener could then receive basic training in the other function later, at the 
discretion of the FSD, but could not be trained in both functions 
immediately upon hire. The new dual-training program is modular in 
design. Thus, FSDs can chose whether newly hired screeners will receive 
instruction in one or both of the screening functions during the initial 
training. In addition, the individual modules can also be used to provide 
recurrent training, such as refreshing checked baggage screening skills for 
a screener who has worked predominately as a passenger screener. TSA 
officials stated that this new approach provides the optimum training 
solution based on the specific needs of each airport and reflects the fact 
that at some airports the FSD does not require all screeners to be fully 
trained in both passenger and checked baggage screening functions. 

Some FSDs, particularly those at smaller airports, have made use of the 
flexibility offered by the modular design of the new course to train 
screeners immediately upon hire in both passenger and checked baggage 
screening functions. Such training up front allows FSDs to use screeners 
for either the passenger or the checked baggage screening function, 
immediately upon completion of basic training. Figure 4 shows that 
58 percent (3,324) of newly hired screeners trained between April 1, 2004, 
and September 1, 2004, had completed the dual-function training. 



Department of Homeland Security, Office of Inspector General: An Evaluation of the 
Transportation Security Administration's Screener Training and Methods of Testing, 
OIG-04-045 (Washington, D.C., Sept. 2004). 
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Figure 4: Newly Hired Screeners Trained as Dual-Function Screeners between April 1, 2004, and September 1, 2004 




Percentage dual-trained 

Percentage trained as passenger screeners only 
Percentage trained as checked baggage screeners only 
Source: GAO analysis of TSA data. 



In April 2004, TSA also provided FSDs with the flexibility to deliver basic 
screener training using local instructors. TSA's Workforce Performance 
and Training Office developed basic screener training internally, and 
initially, contractors delivered all of the basic training. Since then, TSA has 
provided FSDs with the discretion to provide the training using local TSA 
employees or to use contractors. The flexibility to use local employees 
allows FSDs and members of the screener workforce to leverage their 
first-hand screening knowledge and experience and address situations 
unique to individual airports. 20 As of December 10, 2004, TSA had trained 
1,021 local FSD staff (representing 218 airports) in how to instruct the 
dual-function screener training course. TSA officials stated that they 
expect the use of TSA-approved instructors to increase over time. 



For local employees to be approved as instructors by TSA, they must be nominated by an 
FSD and have a current or prior instructor certification by a recognized training and 
development organization or have had at least 2 years of experience as an instructor. In 
addition, local TSA instructors must have successfully completed the course of instruction 
they will be teaching and demonstrate instructional skills by assisting a TSA-approved 
instructor in classroom instruction and monitoring actual classroom instruction. 
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TSA Has Deployed 
Recurrent Screener 
Training and Provided 
Additional Training Tools, 
but Some FSDs Identified 
the Need for More Courses 



TSA has taken several steps to deploy a recurrent screener training 
program, including implementing the Online Learning Center — which 
includes several recurrent training modules developed by TSA — 
implementing a recurrent training requirement, and providing hands-on 
training tools. According to TSA, comprehensive and frequent training is 
key to a screener's ability to detect threat objects. In September 2003, we 
reported that TSA had not fully developed or deployed a recurrent training 
program for passenger screeners. At that time, some FSDs expressed 
concern with the lack of training available to the screener workforce, and 
in the absence of headquarters-provided training, they were developing 
and implementing locally based recurrent training programs. Similar to 
our findings, TSA's April 2004 consultant study reported that most FSDs 
and their staffs were generally unsatisfied with the training support 
provided by headquarters. 21 Specifically, the study found that: 



"Numerous interviews revealed concerns with training curriculum, communication, and 
coordination issues that directly affect security screening. Unsatisfied with the quantity 
and breadth of topics, many Training Coordinators have developed supplementary lectures 
on both security and non-security related topics. These additional lectures. . .have been very 
highly received by screeners." 

In October 2003, TSA introduced the Online Learning Center to provide 
screeners with remote access to self-guided training courses. As of 
September 14, 2004, TSA had provided access to over 550 training courses 
via the Online Learning Center and made the system available via the 
Internet and its intranet. TSA also developed and deployed a number of 
hands-on training modules and associated training tools for screeners at 
airports nationwide. These training modules cover topics including hand- 
wanding and pat-down techniques, physical bag searches, X-ray images, 
prohibited items, and customer service. Additionally, TSA instituted 
another module for the Online Learning Center called Threat in the 
Spotlight, that, based on intelligence TSA receives, provides screeners 
with the latest in threat information regarding terrorist attempts to get 
threat objects past screening checkpoints. Appendix III provides a 
summary of the recurrent training tools TSA has deployed to airports and 
the modules currently under development. 



In April 2004, a TSA-sponsored independent evaluation of screening operations was 
completed by a consultant (Private Screening Operations Performance Evaluation 
Summary Report, BearingPoint, Apr. 16, 2004). The study was designed to evaluate the 
performance of federal and private screening operations. 
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In December 2003, TSA issued a directive requiring screeners to receive 
3 hours of recurrent training per week averaged over a quarter year. One 
hour is required to be devoted to X-ray image interpretation and the other 
2 hours to screening techniques, review of standard operating procedures, 
or other mandatory administrative training, such as ethics and privacy act 
training. 

In January 2004, TSA provided FSDs with additional tools to facilitate and 
enhance screener training. Specifically, TSA provided airports with at least 
one modular bomb set (MBS II) kit — containing components of an 
improvised explosive device — and one weapons training kit, in part 
because screeners had consistently told TSA's OIAPR inspectors that they 
would like more training with objects similar to ones used in covert 
testing. 22 

Although TSA has made progress with the implementation of recurrent 
training, some FSDs identified the need for several additional courses, 
including courses that address more realistic threats. TSA acknowledged 
that additional screener training is needed, and officials stated that the 
agency is in the process of developing new and improved screener 
training, including additional recurrent training modules (see app. III). 



TSA has arranged for leadership training for screening supervisors through 
the Department of Agriculture Graduate School and has developed 
leadership and technical training courses for screening supervisors. 
However, some FSDs reported the need for more training for Screening 
Supervisors and Lead Screeners. The quality of Screening Supervisors has 
been a long-standing concern. In testifying before the 9/11 Commission in 
May 2003, a former FAA Assistant Administrator for Civil Aviation Security 
stated that following a series of covert tests at screening checkpoints to 
determine which were strongest, which were weakest, and why, invariably 
the checkpoint seemed to be as strong or as weak as the supervisor who 
was running it. Similarly, TSA's OIAPR identified a lack of supervisory 
training as a cause for screener covert testing failures. Further, in a July 
2003 internal study of screener performance, TSA identified poor 



The MBS II and weapons training kits were provided to airports to address the identified 
training gap by allowing screeners to see and feel the threat objects that they are looking 
for. These kits contain some of the test objects used by TSA's OAIPR to conduct the covert 
testing. In February 2004, TSA issued guidance to FSDs on use of these kits to conduct 
local screener testing. These guidelines were updated in June 2004. 



TSA Provides Leadership 
and Technical Training for 
Supervisors, but Some 
FSDs Would Like More 
Training for Screening 
Supervisors and Lead 
Screeners 
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supervision at the screening checkpoints as a cause for screener 
performance problems. In particular, TSA acknowledged that many Lead 
Screeners, Screening Supervisors, and Screening Managers did not 
demonstrate supervisory and management skills (i.e., mentoring, coaching, 
and positive reinforcement) and provided little or no timely feedback to 
guide and improve screener performance. In addition, the internal study 
found that because of poor supervision at the checkpoint, supervisors or 
peers were not correcting incorrect procedures, optimal performance 
received little reinforcement, and not enough breaks were provided to 
screeners. A September 2004 report by the DHS OIG supported these 
findings, noting that Screening Supervisors and Screening Managers 
needed to be more attentive in identifying and correcting improper or 
inadequate screener performance. 23 

TSA recognizes the importance of Screener Supervisors and has 
established training programs to enhance their performance and 
effectiveness. In September 2003, we reported that TSA had begun 
working with the Department of Agriculture Graduate School to tailor the 
school's off-the-shelf supervisory course to meet the specific needs of 
Screening Supervisors, and had started training the existing supervisors at 
that time through this course until the customized course was fielded. 
According to TSA's training records, as of September 2004, about 3,800 
Screening Supervisors had completed the course — approximately 
92 percent of current Screening Supervisors. In response to our survey, 
one FSD noted that the supervisory training was long overdue because 
most of the supervisors had no prior federal service or, in some cases, no 
leadership experience. This FSD also noted that "leadership and 
supervisory skills should be continuously honed; thus, the development of 
our supervisors should be an extended and sequential program with 
numerous opportunities to develop skills — not just a one-time class." 

In addition to the Department of Agriculture Graduate School course, 
TSA's Online Learning Center includes over 60 supervisory courses 
designed to develop leadership and coaching skills. In April 2004, TSA 
included in the Online Learning Center a Web-based technical training 
course — required for all Lead Screeners and Screening Supervisors. This 
course covers technical issues, such as resolving alarms at screening 



Department of Homeland Security Office of Inspector General, Audit of Passenger and 
Baggage Screening Procedures at Domestic Airports, OIG-04-37 (Washington, D.C., 
September 2004). 
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checkpoints. TSA introduced this course to the field in March 2004, and 
although the course is a requirement, TSA officials stated that they have 
not set goals for when all Lead Screeners and Screening Supervisors 
should have completed the course. In June 2004, TSA training officials 
stated that a second supervisor technical course was planned for 
development and introduction later in 2004. However, in December 2004, 
the training officials stated that planned funding for supervisory training 
may be used to support other TSA initiatives. The officials acknowledged 
that this would reduce TSA's ability to provide the desired type and level 
of supervisory training to its Lead Screener, Screening Supervisor, and 
Screening Manager staff. TSA plans to revise its plans to provide Lead 
Screener, Screening Supervisor, and Screening Manager training based on 
funding availability. 

Although TSA has developed leadership and technical courses for 
Screening Supervisors, many FSDs, in response to our general survey, 
identified additional types of training needed to enhance screener 
supervision. Table 2 provides a summary of the additional training needs 
that FSDs reported. 



Table 2: Training Needed to Enhance Screener Supervision, as Reported by FSDs 

Percentage of FSDs 



Training topic indicating this need 

Leadership and management 41 

Public speaking/communications 26 

Conflict management 24 

Human resource-related issues (hiring, termination, discipline) 19 

Counseling, mentoring, and coaching 18 

Writing skills 16 

Additional supervisory training 1 3 

Interpersonal skills 13 

Crisis management and incident response 6 

Source: GAO general survey of FSDs. 



TSA training officials stated that the Online Learning Center provides 
several courses that cover these topics. Such courses include 

• Situation Leadership II; 

• Communicating with Difficult People: Handling Difficult Co- Workers; 

• Team Participation: Resolving Conflict in Teams; 
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• Employee Performance: Resolving Conflict; 

• High Impact Hiring; 

• Team Conflict: Overcoming Conflict with Communication; 

• Correcting Performance Problems: Disciplining Employees; 

• Team Conflict: Working in Diversified Teams; 

• Correcting Performance Problems: Identifying Performance Problems; 

• Resolving Interpersonal Skills; 

• Grammar, Skills, Punctuation, Mechanics and Word Usage; and 

• Crisis in Organizations: Managing Crisis Situations. 

TSA training officials acknowledged that for various reasons FSDs might 
not be aware that the supervisory and leadership training is available. For 
example, FSDs at airports without high-speed Internet/intranet access to 
the Online Learning Center might not have access to all of these courses. It 
is also possible that certain FSDs have not fully browsed the contents of 
the Online Learning Center and therefore are not aware that the training is 
available. 24 Furthermore, officials stated that online learning is relatively 
new to government and senior field managers, and some of the FSDs may 
expect traditional instructor-led classes rather than online software. 



Some FSDs responded to our general survey that they faced challenges 
with screeners receiving recurrent training, including insufficient staffing 
to allow all screeners to complete training within normal duty hours and a 
lack of high-speed Internet/intranet connectivity at some training facilities. 
According to our guide for assessing training, to foster an environment 
conducive to effective training and development, agencies must take 
actions to provide sufficient time, space, and equipment to employees to 
complete required training. 25 TSA has set a requirement for 3 hours of 
recurrent training per week averaged over a quarter year, for both full-time 
and part-time screeners. However, FSDs for about 18 percent (48 of 263) 
of the airports in our airport-specific survey reported that screeners 
received less than 9 to 12 hours of recurrent training per month. 26 
Additionally, FSDs for 48 percent (125 of 263) of the airports in the survey 



TSA headquarters training officials stated that the Training Coordinators at airports 
should be aware of the availability of the supervisory and leadership training courses. 

25 GAO, Human Capital: A Guide for Assessing Strategic Training and Development 
Efforts in the Federal Government, GAO-04-546G (Washington, D.C., March 2004). 

26 Methods used by FSDs for determining and reporting recurrent training time have not 
been verified by GAO. 



Some FSDs Reported 
Impediments to Screeners 
Receiving Recurrent 
Training 
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reported that there was not sufficient time for screeners to receive 
recurrent training within regular work hours. 

At 66 percent of those airports where the FSD reported that there was not 
sufficient time for screeners to receive recurrent training within regular 
work hours, the FSDs cited screener staffing shortages as the primary 
reason. We reported in February 2004 that FSDs at 11 of the 15 category X 
airports we visited reported that they were below their authorized staffing 
levels because of attrition and difficulties in hiring new staff. In addition, 
three of these FSDs noted that they had never been successful in hiring up 
to the authorized staffing levels. We also reported in February 2004 that 
FSDs stated that because of staffing shortages, they were unable to let 
screeners participate in training because it affected the FSD's ability to 
provide adequate coverage at the checkpoints. In response to our survey, 
FSDs across all categories of airports reported that screeners must work 
overtime in order to participate in training. A September 2004 DHS OIG 
report recommended that TSA examine the workforce implications of the 
3-hour training requirement and take steps to correct identified 
imbalances in future workforce planning to ensure that all screeners are 
able to meet the recurrent training standard. The 3-hours-per-week 
training standard represents a staff time commitment of 7.5 percent of full- 
time and between 9 and 15 percent of part-time screeners' nonovertime 
working hours. TSA headquarters officials have stated that because the 3- 
hours-per-week requirement is averaged over a quarter, it provides 
flexibility to account for the operational constraints that exist at airports. 
However, TSA headquarters officials acknowledged that many airports are 
facing challenges in meeting the 3-hour recurrent training requirement. 
TSA data for the fourth quarter of fiscal year 2004 reported that 75 percent 
of airports were averaging less than 3 hours of recurrent training per week 
per screener. The current screener staffing model, which is used to 
determine the screener staffing allocations for each airport, does not take 
the 3-hours-per-week recurrent training requirement into account. 27 
However, TSA headquarters officials said that they are factoring this 



The staffing model took into account factors such as the number of screening 
checkpoints and lanes at an airport; originating passengers; projected air carrier service 
increases and decreases during calendar year 2003; and hours needed to accommodate 
some screener training, leave, and breaks. 



Page 22 



GAO-05-457 Screener Training and Performance Measurement Strengthened 



training requirement into their workforce planning efforts, including the 
staffing model currently under development. 28 

Another barrier to providing recurrent training is the lack of high-speed 
Internet/intranet access at some of TSA's training locations. 29 TSA officials 
acknowledged that many of the features of the Online Learning Center, 
including some portions of the training modules and some Online Learning 
Center course offerings, are difficult or impossible to use in the absence of 
high-speed Internet/intranet connectivity. As one FSD put it, "the delayed 
deployment of the high-speed Internet package limits the connectivity to 
TSA HQ for various online programs that are mandated for passenger 
screening operations including screener training." One FSD for a category 
IV airport noted the lack of a high-speed connection for the one computer 
at an airport he oversees made the Online Learning Center "nearly 
useless." 

TSA began deploying high-speed access to its training sites and 
checkpoints in May 2003 and has identified high-speed connectivity as 
necessary in order to deliver continuous training to screeners. TSA's July 
2003 Performance Improvement Study recommended accelerating high- 
speed Internet/intranet access in order to provide quick and systematic 
distribution of information and, thus, reduce uncertainty caused by the 
day-to-day changes in local and national procedures and policy. In 
October 2003, TSA reported plans to have an estimated 350 airports online 
with high-speed connectivity within 6 months. However, in June 2004, TSA 
reported that it did not have the resources to reach this goal. 



In May 2003, TSA hired a contractor to develop a staffing model for its screening 
workforce. TSA officials told us that the contractor completed the staffing model in June 
2004, and all airports now have the capability to use the contractor's stand-alone software. 
TSA completed installation of the software on its intranet in March 2005. This installation 
provides TSA headquarters with access to the staffing models used by airports. The 
Intelligence Reform and Terrorism Prevention Act of 2004 requires TSA to develop and 
submit to the appropriate congressional committees, standards for determining aviation 
security staffing at commercial airports no later than 90 days after December 17, 2004, the 
date of the act's enactment, and GAO to conduct an analysis of these standards (Pub. L. No. 
108-458). As of April 15, 2005, these standards had not been submitted. 

29 High-speed Internet/intranet access is provided by a series of technologies that give users 
the ability to send and receive data at volumes and speeds far greater than Internet access 
over traditional telephone lines. In addition to offering speed, the technology provides a 
continuous, "always on" connection (no need to dial up) and a "two-way" capability, that is, 
the ability to both receive and transmit data at high speeds. 
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TSA records show that as of October 2004, TSA had provided high-speed 
access for training purposes to just 109 airports, where 1,726 training 
computers were fully connected. 30 These 109 airports had an authorized 
staffing level of over 24,900 screeners, meaning that nearly 20,100 
screeners (45 percent of TSA's authorized screening workforce) still did 
not have high-speed Internet/intranet access to the Online Learning Center 
at their training facility. In October 2004, TSA officials stated that TSA's 
Office of Information Technology had selected an additional 16 airport 
training facilities with a total of 205 training computers to receive high- 
speed connectivity by the end of December 2004. As of January 19, 2005, 
TSA was unable to confirm that these facilities had received high-speed 
connectivity. Additionally, they could not provide a time frame for when 
they expected to provide high-speed connectivity to all airport training 
facilities because of funding uncertainties. Furthermore, TSA does not 
have a plan for prioritizing and scheduling the deployment of high-speed 
connectivity to all airport training facilities once funding is available. 
Without a plan, TSA's strategy and timeline for implementing connectivity 
to airport training facilities is unclear. The absence of such a plan limits 
TSA's ability to make prudent decisions about how to move forward with 
deploying connectivity once funding is available. Figure 5 shows the 
percentage of airports reported to have high-speed connectivity for their 
training computers by category of airport as of October 2004. 



30 TSA defines/^% connected as a training computer with the new network image installed 
and connected to the TSA broadband network. 



Page 24 



GAO-05-457 Screener Training and Performance Measurement Strengthened 



Figure 5: Percentage of Airports Reported to Have High-Speed Connectivity for Training Purposes as of October 2004 
Airport category: x I II III IV All airports 




Total number 

of airports: 21 61 51 124 197 454 

Airports with high-speed connectivity 
I Airports without high-speed connectivity 
Source: GAO analysis of TSA data. 

Note: These data show airports with high-speed connectivity on training computers. Some airports 
have high-speed connectivity, but for purposes other than training. 

To mitigate airport connectivity issues in the interim, on April 1, 2004, TSA 
made the Online Learning Center courses accessible through public 
Internet connections, which enable screeners to log on to the Online 
Learning Center from home, a public library, or other locations. However, 
TSA officials stated that the vast majority of screeners who have used the 
Online Learning Center have logged in from airports with connectivity at 
their training facilities. TSA also distributes new required training 
products using multiple delivery channels, including written materials and 
CD-ROMs for those locations where access to the Online Learning Center 
is limited. Specifically, TSA officials stated that they provided airports 
without high-speed connectivity with CD-ROMs for the 50 most commonly 
used optional commercial courseware titles covering topics such as 
information technology skills, customer service, and teamwork. 
Additionally, officials stated that as technical courses are added to the 
Online Learning Center, they are also distributed via CD-ROM and that 
until full connectivity is achieved, TSA will continue to distribute new 
training products using multiple delivery channels. 
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TSA Lacks Adequate 
Controls to Provide 
Reasonable 
Assurance That 
Screeners Receive 
Required Training 



Because of a lack of internal controls, TSA cannot provide reasonable 
assurance that screeners are completing required training. First, TSA 
policy does not clearly define responsibility for ensuring that screeners 
have completed all required training. Additionally, TSA has no formally 
defined policies or procedures for documenting completion of remedial 
training, or a system designed to facilitate review of this documentation 
for purposes of monitoring. Further, TSA headquarters does not have 
formal policies and procedures for monitoring completion of basic training 
and lacks procedures for monitoring recurrent training. Finally, at airports 
without high-speed connectivity, training records must be entered 
manually, making it challenging for some airports to keep accurate and up- 
to-date training records. 



TSA Policy Does Not 
Clearly Define 
Responsibility for 
Ensuring That Screeners 
Are in Compliance with 
Training Requirements 



TSA's current guidance for FSDs regarding the training of the screener 
workforce does not clearly identify responsibility for tracking and 
ensuring compliance with training requirements. In a good control 
environment, areas of authority and responsibility are clearly defined and 
appropriate lines of reporting are established. 31 In addition, internal 
control standards also require that responsibilities be communicated 
within an organization. The Online Learning Center provides TSA with a 
standardized, centralized tool capable of maintaining all training records 
in one system. It replaces an ad hoc system previously used during initial 
rollout of federalized screeners in which contractors maintained training 
records. A February 2004 management directive states that FSDs are 
responsible for ensuring the completeness, accuracy, and timeliness of 
training records maintained in the Online Learning Center for their 
employees. For basic and recurrent training, information is to be entered 
into the Online Learning Center within 30 days of completion of the 
training activity. However, the directive does not clearly identify who is 
responsible for ensuring that employees comply with training 
requirements. Likewise, a December 2003 directive requiring that 
screeners complete 3 hours of training per week averaged over a quarter 
states that FSDs are responsible for ensuring that training records for each 
screener are maintained in the Online Learning Center. Although both 
directives include language that requires FSDs to ensure training records 
are maintained in the Online Learning Center, neither specifies whether 
FSDs or headquarters officials are responsible for ensuring compliance 
with the basic, recurrent, and remedial training requirements. Even so, 
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TSA headquarters officials told us that FSDs are ultimately responsible for 
ensuring screeners receive required training. However, officials provided 
no documentation clearly defining this responsibility. Without a clear 
designation of responsibility for monitoring training completion, this 
function may not receive adequate attention, leaving TSA unable to 
provide reasonable assurance that its screening workforce receives 
required training. In April 2005, TSA officials responsible for training 
stated that they were updating the February 2004 management directive on 
training records to include a specific requirement for FSDs to ensure that 
screeners complete required training. They expect to release the revised 
directive in May 2005. 



TSA Lacks Formal Policies 
and Procedures for 
Monitoring Completion of 
Required Training 



TSA has not established and documented policies and procedures for 
monitoring completion of basic and recurrent training. Internal control 
standards advise that internal controls should be designed so that 
monitoring is ongoing and ingrained in agency operations. 32 However, TSA 
headquarters officials stated that they have no formal policy for 
monitoring screeners' completion of basic training. They also stated that 
they have neither informal nor formal procedures for monitoring the 
completion of screeners' recurrent training requirements, and 
acknowledged that TSA policy does not address what is to occur if a 
screener does not meet the recurrent training requirement. Officials 
further stated that individual FSDs have the discretion to determine what 
action, if any, to take when screeners do not meet this requirement. 

In July 2004, TSA training officials stated that headquarters staff recently 
began running a report in the Online Learning Center to review training 
records to ensure that newly hired screeners had completed required basic 
training. In addition, they stated that in June 2004, they began generating 
summary-level quarterly reports from the Online Learning Center to 
quantify and analyze hours expended for recurrent screener training. 
Specifically, TSA training officials stated that reports showing airport-level 
compliance with the 3-hour recurrent requirement were generated for the 
third and fourth quarters of fiscal year 2004 and delivered to the Office of 
Aviation Operations for further analysis and sharing with the field. 
However, Aviation Operations officials stated that they did not use these 
reports to monitor the status of screener compliance with the 3-hour 
recurrent training requirement and do not provide them to the field unless 
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requested by an FSD. TSA training officials said that while headquarters 
intends to review recurrent training activity on an ongoing basis at a 
national and airport level, they view FSDs and FSD training staff as 
responsible for ensuring that individuals receive all required training. 
Further, they acknowledged that weaknesses existed in the reporting 
capability of the Online Learning Center and stated that they plan to 
upgrade the Online Learning Center with improved reporting tools by the 
end of April 2005. Without clearly defined policies and procedures for 
monitoring the completion of training, TSA lacks a structure to support 
continuous assurance that screeners are meeting training requirements. 



TSA has not established clear policies and procedures for documenting 
completion of required remedial training. The Standards for Internal 
Control state that agencies should document all transactions and other 
significant events and should be able to make this documentation readily 
available for examination. 33 A TSA training bulletin dated October 15, 2002, 
specifies that when remedial training is required, FSDs must ensure the 
training is provided and a remedial training reporting form is completed 
and maintained with the screener's local records. 34 However, when we 
asked to review these records, we found confusion as to how and where 
they were to be maintained. TSA officials stated that they are waiting for a 
decision regarding how to maintain these records because of their 
sensitive nature. In the meantime, where and by whom the records should 
be maintained remains unclear. 

In September 2004, officials from TSA's OIAPR — responsible for 
conducting covert testing — stated that they maintain oversight to ensure 
screeners requiring remedial training receive required training by 
providing a list of screeners that failed covert testing and therefore need 
remedial training to TSA's Office of Aviation Operations. Aviation 
Operations is then to confirm via memo that each of the screeners has 
received the necessary remedial training and report back to OIAPR. 
Accordingly, we asked TSA for all Aviation Operations memos confirming 
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34 Consistent with ATSA, TSA requires remedial training for any passenger or baggage 
screener who fails an operational (covert) test and prohibits screeners from performing the 
screening function related to the test they failed until they successfully complete the 
training. 



TSA Lacks Clearly Defined 
Policies and Procedures 
for Documenting 
Completion of Remedial 
Training 
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completion of remedial training, but we were only able to obtain 1 of the 
12 memos. 35 

In addition, during our review, we asked to review the remedial training 
reporting forms at five airports to determine whether screeners received 
required training, but we encountered confusion about requirements for 
maintaining training records and inconsistency in record keeping on the 
part of local TSA officials. Because of the unclear policies and procedures 
for recording completion of remedial training, TSA does not have adequate 
assurance that screeners are receiving legislatively mandated remedial 
training. 



Lack of High-Speed 
Connectivity Limits TSA's 
Ability to Document and 
Track the Completion of 
Screener Training 



Although training computers with high-speed Internet/intranet 
connectivity automatically record completion of training in the Online 
Learning Center, airports without high-speed access at their training 
facility must have these records entered manually. The February 2004 
management directive that describes responsibility for entering training 
records into the Online Learning Center also established that all TSA 
employees are required to have an official TSA training record in the 
Online Learning Center that includes information on all official training 
that is funded wholly or in part with government funds. Without high- 
speed access, TSA officials stated that it can be a challenge for airports to 
keep the Online Learning Center up to date with the most recent training 
records. TSA headquarters officials further stated that when they want to 
track compliance with mandatory training such as ethics or civil rights 
training, they provide the Training Coordinators with a spreadsheet on 
which to enter the data rather than relying on the Online Learning Center. 
As one FSD told us, without high-speed connectivity at several of the 
airports he oversees, "this is very time consuming and labor intensive and 
strains my limited resources." The difficulty that airports encounter in 
maintaining accurate records when high-speed access is absent could 
compromise TSA's ability to provide reasonable assurance that screeners 
are receiving mandated basic and remedial training. 



According to TSA officials, between September 2002 and December 8, 2004, the Office of 
Internal Affairs and Program Review issued 12 memorandums to Aviation Operations that 
identified screeners requiring remedial training based on covert testing conducted from 
September 2002 through September 30, 2004. 
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Progress Has Been 
Made in Implementing 
Tools to Measure and 
Enhance Screener 
Performance, but Key 
Performance Targets 
Have Not Been 
Finalized 



TSA has improved its efforts to measure and enhance screener 
performance. However, these efforts have primarily focused on passenger 
screening rather than checked baggage screening, and TSA has not yet 
finalized performance targets for several key performance measures. For 
example, TSA has increased the amount of covert testing it performs at 
airports. These tests have identified that, overall, weaknesses and 
vulnerabilities continue to exist in the passenger and checked baggage 
screening systems. TSA also enabled FSDs to conduct local covert testing, 
fully deployed the Threat Image Projection (TIP) system to passenger 
screening checkpoints at commercial airports nationwide, and completed 
the 2003/2004 annual screener recertification program for all eligible 
screeners. However, not all of these performance measurement and 
enhancement tools are available for checked baggage screening. 
Specifically, TIP is not currently operational at checked baggage screening 
checkpoints, and the recertification program does not include an image 
recognition component for checked baggage screeners. However, TSA is 
taking steps to address the overall imbalance in passenger and checked 
baggage screening performance data, including working toward 
implementing TIP for checked baggage screening and developing an image 
recognition module for checked baggage screener recertification. To 
enhance screener and screening system performance, TSA has also 
conducted a passenger screener performance improvement study and 
subsequently developed an improvement plan consisting of multiple action 
items, many of which TSA has completed. However, TSA has not 
conducted a similar study for checked baggage screeners. In addition, TSA 
has established over 20 performance measures for the passenger and 
checked baggage screening systems as well as two performance indexes 
(one for passenger and one for checked baggage screening). However, 
TSA has not established performance targets for each of the component 
indicators within the indexes, such as covert testing. According to The 
Office of Management and Budget, performance goals are target levels of 
performance expressed as a measurable objective, against which actual 
achievement can be compared. Performance goals should incorporate 
measures (indicators used to gauge performance); targets (characteristics 
that tell how well a program must accomplish the measure), and time 
frames. Without these targets, TSA's performance management system, 
and these performance indexes, specifically, may not provide the agency 
with the complete information necessary to assess achievements and 
make decisions about where to direct performance improvement efforts. 
Although TSA has not yet established performance targets for each of the 
component indicators, TSA plans to finalize performance targets for the 
indicators by the end of fiscal year 2005. 
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TSA Has Increased Its 
Covert Testing and 
Allowed Local Covert 
Testing at Passenger 
Screening Checkpoints 



TSA headquarters has increased the amount of covert testing it performs 
and enabled FSDs to conduct additional local covert testing at passenger 
screening checkpoints. TSA's OIAPR conducts unannounced covert tests 
of screeners to assess their ability to detect threat objects and to adhere to 
TSA-approved procedures. These tests, in which undercover OIAPR 
inspectors attempt to pass threat objects through passenger screening 
checkpoints and in checked baggage, are designed to measure 
vulnerabilities in passenger and checked baggage screening systems and 
to identify systematic problems affecting performance of screeners in the 
areas of training, policy, and technology. 36 TSA considers its covert testing 
as a "snapshot" of a screener's ability to detect threat objects at a 
particular point in time and as one of several indicators of system wide 
screener performance. 

OIAPR conducts tests at passenger screening checkpoints and checked 
baggage screening checkpoints. According to OIAPR, these tests are 
designed to approximate techniques terrorists might use. These covert test 
results are one source of data on screener performance in detecting threat 
objects as well as an important mechanism for identifying areas in 
passenger and checked baggage screening needing improvement. In 
testimony before the 9/11 Commission, the Department of Transportation 
Inspector General stated that emphasis must be placed on implementing 
an aggressive covert testing program to evaluate operational effectiveness 
of security systems and equipment. 37 



Between September 10, 2002, and September 30, 2004, OIAPR conducted a 
total of 3,238 covert tests at 279 different airports. In September 2003, we 
reported that OIAPR had conducted limited covert testing but planned to 
double the amount of tests it conducted during fiscal year 2004, based on 
an anticipated increase in its staff from about 100 full-time equivalents to 
about 200 full-time equivalents. TSA officials stated that based on budget 
constraints, OIAPR's fiscal year 2004 staffing authorization was limited to 



OIAPR designs its covert testing methods based, in part, on intelligence regarding the 
most recent threats. 

37 Statement before the National Commission on Terrorist Attacks upon the United States, 
Statement of the Honorable Kenneth M. Mead, Inspector General, U.S. Department of 
Transportation, May 22, 2003. 
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183 full-time-equivalents, 38 of which about 60 are located in the field. 39 
Despite a smaller than expected staff increase, by the end of the second 
quarter of fiscal year 2004, OIAPR had already surpassed the number of 
tests it performed during fiscal year 2003, as shown in table 3. 



Covert testing is an ancillary duty and not a full-time assignment for the majority of 
OIAPR staff. According to OIAPR, of the approximately 123 full-time-equivalent positions in 
headquarters, 14 are dedicated fully to the covert testing program, which includes covert 
testing of all modes of transportation, not just airports. These 14 full-time-equivalents are in 
OIAPR's Special Operations group and form the core of team leaders for the covert testing 
trips. In addition, two full-time-equivalents from OIAPR's Office of Program Analysis 
support the covert testing program full-time in terms of data analysis, report writing, and 
quality assurance. The remaining OIAPR staff in headquarters and the field are responsible 
for conducting criminal and noncriminal investigations of employee misconduct; 
conducting program reviews, inspections, and special inquiries into security incidents; and 
managing OIAPR. 

39 TSA established five mission support centers staffed with OIAPR, training, and other 
personnel. These centers are located in Atlanta, Dallas, Detroit, Philadelphia, and San 
Francisco. In fiscal year 2004, OIAPR began to use field staff to support its covert testing 
activities. In addition, OIAPR has 12 investigators located at seven airports. 
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Table 3: Checkpoint and Checked Baggage Tests Conducted by OIAPR, 
September 10, 2002-September 30, 2004 



Testing period 
fiscal 

year/quarter 


Airports 3 


Passenger 
screening 
checkpoint tests 


Checked 
baggage tests' 1 


Total tests 


2002 fourth 


2 


30 


3 


33 


2003 first 


14 


120 


1 


121 


2003 second 


31 


231 


27 


258 


2003 third 


28 


198 


19 


217 


2003 fourth 


25 


217 


23 


240 


2004 first 


41 


171 


110 


281 


2004 second 


111 


770 


182 


952 


2004 third 


56 


379 


102 


481 


2004 fourth 


64 


527 


128 


655 


Total 




2,643 


595 


3,238 



Source: GAO analysis of OIAPR data. 

Note: Some airports have been tested more than once. 

a OIAPR conducted covert testing at three additional airports in September and October 2002. 
However, at the time of the testing, federal screeners had not yet been deployed to these airports. 
We excluded these tests from our analysis. 

b TSA did not begin reporting the results of checked baggage tests until January 2003. However, four 
of these tests were conducted in September and October 2002. 

In October 2003, OIAPR committed to testing between 90 and 150 airports 
by April 2004 as part of TSA's short-term screening performance 
improvement plan. OAIPR officials stated that this was a onetime goal to 
increase testing. This initiative accounts for the spike in testing for the 
second quarter of fiscal year 2004. 

OIAPR has created a testing schedule designed to test all airports at least 
once during a 3-year time frame. Specifically, the schedule calls for OIAPR 
to test all category X airports once a year, category I and II airports once 
every 2 years, and category III and IV airports at least once every 3 years. 

In September 2003 and April 2004, we reported that TSA covert testing 
results had identified weaknesses in screeners' ability to detect threat 
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objects. 40 More recently, in April 2005, we, along with the DHS OIG, 
identified that screener performance continued to be a concern. 
Specifically, our analysis of TSA's covert testing results for tests 
conducted between September 2002 and September 2004 identified that 
overall, weaknesses still existed in the ability of screeners to detect threat 
objects on passengers, in their carry-on bags, and in checked baggage. 41 
Covert testing results in this analysis cannot be generalized either to the 
airports where the tests were conducted or to airports nationwide. 42 These 
weaknesses and vulnerabilities were identified at airports of all sizes, at 
airports with federal screeners, and airports with private-sector screeners. 
For the two-year period reviewed, overall failure rates for covert tests 
(passenger and checked baggage) conducted at airports using private- 
sector screeners were somewhat lower than failure rates for the same 
tests conducted at airports using federal screeners for the airports tested 
during this period. 43 Since these test results cannot be generalized as 
discussed above, each airport's test results should not be considered a 
comprehensive measurement of the airport's performance or any 
individual screener's performance in detecting threat objects, or in 
determining whether airports with private sector screeners performed 
better than airports with federal screeners. 

On the basis of testing data through September 30, 2004, we determined 
that OIAPR had performed covert testing at 61 percent of the nation's 
commercial airports. TSA has until September 30, 2005, to test the 
additional 39 percent of airports and meet its goal of testing all airports 



GAO, Airport Passenger Screening: Preliminary Observations on Progress Made and 
Challenges Remaining, GAO-03-1173 (Washington, D.C.: Sept. 24, 2003). GAO, Aviation 
Security: Private Screening Contractors Have Little Flexibility to Implement Innovative 
Approaches, GAO-04-544T (Washington, D.C.: Apr 22, 2004). 

41 Results of TSA's covert testing of passenger and checked baggage screening are classified 
and are discussed in a separate classified report, GAO, Results of Transportation Security 
Administration's Covert Testing for Passenger and Checked Baggage Screening for 
September 2002 through September 2004, GAO-05-437C (Washington, D.C.: Apr. 5, 2005). 

42 Test results cannot be generalized because sample tests were not identified using the 
principles of probability sampling. In a probability sample to assess screener detection of 
threat objects, each screening of a passenger or baggage would have to have a chance of 
being selected. A well-designed probability sample would enable failure rates to be 
generalized to all airports. However, for cost and operational reasons, probability sampling 
may not be feasible for passenger and checked baggage screening because it would require 
a very large sample size and an exhaustive examination of each sampled passenger or 
baggage to determine if there was a threat object to detect. 

43 Test failure rates are classified and have been excluded from this report. 
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within 3 years. Although officials stated that they have had to divert 
resources from airport testing to conduct testing of other modes and that 
testing for other modes of transportation may affect their ability to 
conduct airport testing, they still expect to meet the goal. 

In February 2004, TSA provided protocols to help FSDs conduct their own 
covert testing of local airport passenger screening activities — a practice 
that TSA had previously prohibited. 44 Results of local testing using these 
protocols are to be entered into the Online Learning Center. This 
information, in conjunction with OAIPR covert test results and TIP threat 
detection results, is intended to assist TSA in identifying specific training 
and performance improvement efforts. In February 2005, TSA released a 
general procedures document for local covert testing at checked baggage 
screening locations. 

TSA officials said that they had not yet begun to use data from local covert 
testing to identify training and performance needs because of difficulties 
in ensuring that local covert testing is implemented consistently 
nationwide. These officials said that after a few months of collecting and 
assessing the data, they will have a better idea of how the data can be 
used. 



TSA Has Reinstated the 
Threat Image Projection 
System and Plans to Use 
Its Data to Improve the 
Screening System 



TSA has nearly completed the reactivation of the TIP system at airports 
nationwide and plans to use data it is collecting to improve the 
effectiveness of the passenger screening system. 45 TIP is designed to test 
passenger screeners' detection capabilities by projecting threat images, 
including guns, knives, and explosives, onto bags as they are screened 
during actual operations. Screeners are responsible for identifying the 
threat image and calling for the bag to be searched. Once prompted, TIP 
identifies to the screener whether the threat is real and then records the 
screener's performance in a database that could be analyzed for 



The local covert testing protocols were updated in June 2004 and August 2004 to provide 
information on alternative testing methods. 

45 TIP is not yet operational at one airport because of construction at the screening 
checkpoint to prepare for its installation. However, the TIP-ready X-ray machines have 
already been procured for the airport and are expected to be installed once the 
construction issues have been resolved. 
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performance trends. 46 TSA is evaluating the possibility of developing an 
adaptive functionality to TIP. Specifically, as individual screeners become 
proficient in identifying certain threat images, such as guns or knives, they 
will receive fewer of those images and more images that they are less 
proficient at detecting, such as improvised explosive devices. 

TIP was activated by FAA in 1999 with about 200 threat images, but it was 
shut down immediately following the September 11 terrorist attacks 
because of concerns that it would result in screening delays and panic, as 
screeners might think that they were actually viewing threat objects. In 
October 2003, TSA began reactivating and expanding TIP. In April 2004, 
we reported that TSA was reactivating TIP with an expanded library of 
2,400 images at all but one of the more than 1,800 checkpoint lanes 
nationwide. To further enhance screener training and performance, TSA 
also plans to develop at least an additional 50 images each month. 

Despite these improvements, TIP is not yet available for checked baggage 
screening. In April 2004, we reported that TSA officials stated that they 
were working to resolve technical challenges associated with using TIP for 
checked baggage screening on EDS machines and have started EDS TIP 
image development. The DHS OIG reported in September 2004 that TSA 
plans to implement TIP on all EDS machines at checked baggage stations 
nationwide in fiscal year 2005. However, in December 2004, TSA officials 
stated that because of severe budget reductions, TSA will be unable to 
begin implementing a TIP program for checked baggage in fiscal year 2005. 
They did not specify when such a program might begin. 

TSA plans to use TIP data to improve the passenger screening system in 
two ways. First, TIP data can be used to measure screener threat detection 
effectiveness by different threats. Second, TSA plans to use TIP results to 
help identify specific recurrent training needs within and across airports 
and to tailor screeners' recurrent training to focus on threat category areas 
that indicate a need for improvement. TSA considers February 2004 as the 
first full month of TIP reporting with the new library of 2,400 images. TSA 
began collecting these data in early March 2004 and is using the data to 
determine more precisely how they can be used to measure screener 



46 The TIP database records both the TIP hit rate and TIP false alarm rate. These two results 
are used to determine the probability of detection and probability of false alarm, which 
determine overall TIP performance. 
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performance in detecting threat objects and to determine what the data 
identify about screener performance. 

TSA does not currently plan to use TIP data as an indicator of individual 
screener performance because TSA does not believe that TIP by itself 
adequately reflects a screener's performance. Nevertheless, in April 2004, 
TSA gave FSDs the capability to query and analyze TIP data in a number of 
ways, including by screener, checkpoint, and airport. FSDs for over 
60 percent of the airports included in our airport-specific survey stated 
that they use or plan to use TIP data as a source of information in their 
evaluations of individual screener performance. Additionally, FSDs for 
50 percent of the airports covered in our survey reported using data 
generated by TIP to identify specific training needs for individual 
screeners. 

In September 2004, the DHS OIG reported that TSA is assessing the cost 
and feasibility of modifying TIP so that it recognizes and responds to 
specific threat objects with which individual screeners are most and least 
competent in detecting, over time. This feature would increase the utility 
of TIP as a training tool. The DHS OIG also reported that TSA is 
considering linking TIP over a network, which would facilitate TSA's 
collection, analysis, and information-sharing efforts around TIP user 
results. The report recommended that TSA continue to pursue each of 
these initiatives, and TSA agreed. However, in December 2004, TSA 
officials stated that the availability of funding will determine whether or 
not they pursue these efforts further. 



TSA Has Completed Its 
First Round of Screener 
Recertification Testing, but 
Testing for Checked 
Baggage Screeners Is Not 
as Comprehensive as for 
Passenger Screeners 



TSA has completed its first round of the screener recertification program, 
and the second round is now under way. However, TSA does not currently 
include an image recognition component in the test for checked baggage 
screener recertification. ATSA requires that each screener receive an 
annual proficiency review to ensure he or she continues to meet all 
qualifications and standards required to perform the screening function. In 
September 2003, we reported that TSA had not yet implemented this 
requirement. 47 To meet this requirement, TSA established a recertification 
program, and it began recertification testing in October 2003 and 
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Page 37 GAO-05-457 Screener Training and Performance Measurement Strengthened 



completed the testing in March 2004. 48 The first recertification program 
was composed of two assessment components, one of screeners' 
performance and the other of screeners' knowledge and skills. During the 
performance assessment component of the recertification program, 
screeners are rated on both organizational and individual goals, such as 
maintaining the nation's air security, vigilantly carrying out duties with 
utmost attention to tasks that will prevent security threats, and 
demonstrating the highest levels of courtesy to travelers to maximize their 
levels of satisfaction with screening services. The knowledge and skills 
assessment component consists of three modules: (1) knowledge of 
standard operating procedures, (2) image recognition, and (3) practical 
demonstration of skills. Table 4 provides a summary of these three 
modules. 



Table 4: Modules Included in Recertification Knowledge and Skills Assessment 



Testing modules 



Description 



Knowledge of standard operating 
procedures 



Computerized 50-question multiple-choice test. It 
is either passenger- or baggage-specific. 



Image recognition 



Computerized test that consists of 100 images 
and is used to evaluate a screener's skill and 
ability in detecting threat or prohibited objects 
within X-ray images. 



Practical demonstration of skills 



Hands-on simulated work sample to evaluate a 
screener's knowledge, skills, and ability when 
performing specific screener tasks along with 
ability to provide customer service. 



Source: TSA. 



To be recertified, screeners must have a rating of "met" or "exceeded" 
standards on their annual performance assessments and have passed each 
of the applicable knowledge and skills modules. Screeners that failed any 
of the three modules were to receive study time or remedial training as 
well as a second opportunity to take and pass the modules. Screeners who 
failed on their second attempt were to be removed from screening duties 
and subject to termination. Screeners could also be terminated for 
receiving a rating of below "met" standards. 



Some screeners, such as those on extended leave, leave without pay, military leave, or 
leave because of an on-the-job injury were not tested. These screeners are retested as they 
return to work. 
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TSA completed its analysis of the recertification testing and performance 
evaluations in May 2004. TSA's analysis shows that less than 1 percent of 
screeners subject to recertification failed to complete this requirement. 
Figure 6 shows the recertification results. 



Figure 6: Screener Recertification Results, October 2003-March 2004 




42,690 Screeners recertified 
352 Screeners not recertified 



Screeners recertified 



Screeners not recertified 
Source: GAO analysis of TSA data. 

Across all airports screeners performed well on the recertification testing. 
Over 97 percent of screeners passed the standard operating procedures 
test on their first attempt. Screeners faced the most difficulty on the 
practical demonstration of skills component. However, following 
remediation, 98.6 percent of the screeners who initially failed this 
component passed on their second attempt. Table 5 shows the results of 
the recertification testing by module. 



Table 5: Screener Recertification Module Testing Percentage Pass Rates, 
October 2003— March 2004 





Standard operating 
procedures 


Image recognition 


Practical skills 
demonstration 


First attempt 


97.4 


96.0 


75.2 


Retest 


96.8 


84.3 


98.6 


Overall 


99.9 


99.5 


99.7 



Source: TSA. 
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As shown in table 6, screeners hired as checked baggage screeners were 
not required to complete the image recognition module in the first round 
of the recertification testing. 49 



Table 6: Recertification Testing Modules by Screening Function 




Checked 


Testing modules Passenger 


baggage 


Knowledge of standard operating procedures V 


V 


Image recognition V 


Practical demonstration of skills V 


V 



Source: TSA. 



In addition, during the first year of recertification testing, which took place 
from October 2003 through May 2004, dual-function screeners who were 
actively working as both passenger and checked baggage screeners were 
required to take only the recertification test for passenger screeners. They 
were therefore not required to take the recertification testing modules 
required for checked baggage, even though they worked in that capacity. 50 

TSA began implementing the second annual recertification testing in 
October 2004 and plans to complete it no later than June 2005. This 
recertification program includes components for dual-function screeners. 
However, TSA still has not included an image recognition module for 
checked baggage screeners — which would include dual-function screeners 
performing checked baggage screening. TSA officials stated that a decision 
was made to not include an image recognition module for checked 
baggage screeners during this cycle because not all checked baggage 
screeners would have completed training on the onscreen resolution 
protocol by the time recertification testing was conducted at their 
airports. 51 In December 2004, TSA officials stated that they plan on 



According to TSA records, between October 2003 and May 2004, 13,516 screeners 
completed the recertification testing as baggage screeners. 

50 As of January 7, 2005, TSA reported that its workforce included approximately 25,947 
dual-trained screeners who were certified to serve as passenger or baggage screeners. 

51 TSA's onscreen resolution protocol requires that when an EDS machine alarm goes off, 
indicating the possibility of explosives, TSA screeners, by reviewing computer-generated 
images of the inside of the bag, attempt to determine whether or not a suspect item or 
items are in fact explosive materials. If the screener is unable to make this determination, 
the bag is diverted from the main conveyor belt into an area where it receives a secondary 
screening by a screener with an ETD machine. 
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developing an image recognition module for checked baggage and dual- 
function screeners, and that this test should be available for next year's 
recertification program. The development and implementation of the 
image recognition test will be contingent, they stated, upon the availability 
of funds. 



TSA Has Identified and 
Implemented Efforts to 
Enhance Screener 
Performance, but These 
Efforts Primarily Focused 
on Passenger Screeners 



TSA has implemented a number of improvements designed to enhance 
screener performance, based on concerns it identified in a July 2003 
Passenger Screener Performance Improvement Study and 
recommendations from OIAPR. To date, however, these efforts have 
primarily focused on the performance of passenger screeners, and TSA 
has not yet undertaken a comparable performance study for checked 
baggage screeners. The Passenger Screener Performance Improvement 
Study relied in part on the findings of OIAPR's covert testing. At the time 
the study was issued, OIAPR had conducted fewer than 50 tests of 
checked baggage screeners. The July 2003 study focused on and included 
numerous recommendations for improving the performance of passenger 
screeners, but recommended waiting to analyze the performance of 
checked baggage screeners until some time after implementation of the 
recommendations, some of which TSA indicated, also applied to checked 
baggage screeners. TSA officials told us that this analysis has been 
postponed until they have reviewed the impact of implementing the 
recommendations on passenger screening performance. 

In October 2003, to address passenger screener performance deficiencies 
identified in the Screener Performance Improvement Study, TSA 
developed a Short-Term Screening Performance Improvement Plan. This 
plan included specific action items in nine broad categories — such as 
enhance training, increase covert testing, finish installing TIP, and 
expedite high-speed connectivity to checkpoints and training computers — 
that TSA planned to pursue to provide tangible improvements in passenger 
screener performance and security (see app. IV for additional information 
on the action items). In June 2004, TSA reported that it had completed 57 
of the 62 specific actions. As of December 2004, two of these actions still 
had not been implemented — full deployment of high-speed connectivity 
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and a time and attendance package — both of which continue to be 
deferred pending the identification of appropriate resources. 52 

In addition to the Performance Improvement Study and corresponding 
action plans, TSA's OIAPR makes recommendations in its reports on 
covert testing results. These recommendations address deficiencies 
identified during testing and are intended to improve screening 
effectiveness. As of December 2004, OIAPR had issued 18 reports to TSA 
management on the results of its checkpoint and checked baggage covert 
testing. 53 These reports include 14 distinct recommendations, 54 some of 
which were included in TSA's screener improvement action plan. All but 
two of these reports included recommendations on corrective actions 
needed to enhance the effectiveness of passenger and checked baggage 
screening. 



TSA Has Established 
Screening Performance 
Measures and Indexes but 
Has Not Established Key 
Performance Targets 



TSA has established performance measures, indexes, and targets for the 
passenger and checked baggage screening systems, but has not 
established targets for the various components of the screening indexes. 
The Government Performance and Results Act of 1993 provides, among 
other things, that federal agencies establish program performance 
measures, including the assessment of relevant outputs and outcomes of 
each program activity. 55 Performance measures are meant to cover key 
aspects of performance and help decision makers to assess program 
accomplishments and improve program performance. A performance 
target is a desired level of performance expressed as a tangible, 



TSA officials stated that in early fiscal year 2005, TSA's Office of Information Technology 
committed to extending high-speed connectivity to an additional 16 locations, but is still 
awaiting funding to further expand network coverage and to extend the network to 
checkpoints. 

53 OIAPR has issued its reports to the TSA Administrator; the TSA Administrator's Chief of 
Staff; Associate Undersecretary for Aviation Operations; Associate Undersecretary for 
Workforce, Performance and Training; Assistant Secretary and Chief Technology Officer; 
Assistant Administrator and Chief Support Systems Officer; Chief Operating Officer, the 
Office of Transportation Security and Intelligence, and the Office of Security Policy. The 
report recommendations are directed to the office(s) responsible for taking the corrective 
action. 

54 Some recommendations appear repeatedly in multiple reports issued by OIAPR. 

55 According to the Government Performance and Results Act, the Office of Management 
and Budget, and GAO, outcomes assess actual results as compared with the intended 
results or consequences that occur from carrying out a program or activity. Outputs count 
the goods and services produced by a program or organization. 
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measurable objective, against which actual achievement will be compared. 
By analyzing the gap between target and actual levels of performance, 
management can target those processes that are most in need of 
improvement, set improvement goals, and identify appropriate process 
improvements or other actions. 

An April 2004 consultant study commissioned by TSA found that FSDs and 
FSD staffs generally believed the lack of key performance indicators 
available to monitor passenger and checked baggage screening 
performance represented a significant organizational weakness. Since 
then, TSA has established over 20 performance measures for the 
passenger and checked baggage screening systems. For example, TSA 
measures the percentage of screeners meeting a threshold score on the 
annual recertification testing on their first attempt, the percentage of 
screeners scoring above the national standard level on TIP performance, 
and the number of passengers screened, by airport category. 

TSA also has developed two performance indexes to measure the 
effectiveness of the passenger and checked baggage screening systems. 56 
These indexes measure overall performance through a composite of 
indicators and are derived by combining specific performance measures 
relating to passenger and checked baggage screening, respectively. 
Specifically, these indexes measure the effectiveness of the screening 
systems through machine probability of detection and covert testing 
results; 57 efficiency through a calculation of dollars spent per passenger or 
bag screened; and customer satisfaction through a national poll, customer 
surveys, and customer complaints at both airports and TSA's national call 
center. According to TSA officials, the agency has finalized targets for the 
two overall indexes, but these targets have not yet been communicated 
throughout the agency. Further, TSA plans to provide the FSDs with only 
the performance index score, not the value of each of the components, 
because the probabilities of detection are classified as secret and TSA is 
concerned that by releasing components, those probabilities could be 



56 TSA analyzed data from fiscal years 2003 and 2004 to establish baselines and establish 
performance targets. 

57 According to TSA, the machine probabilities of detection are established by the 
certification standards for each particular model of machines, and machines are not 
deployed unless they have met those standards. 
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deduced. 58 Table 7 summarizes the components of the performance 
indexes developed by TSA. 



Table 7: Components of TSA's Performance Indexes 


Performance index 


Components 


Weight 
(percent) 


FY 2005-2010 
performance 
target 
(1-5 scale) 


Passenger screening 
program 


• Machine probability of 
detection x person 
probability of detection 
(covert testing) 


50 






• Cost per person screened 


25 


3.3 




• Consumer satisfaction 


25 




Checked baggage 
screening program 


• Machine probability of 
detection x person 
probability of detection 
(covert testing) 


50 






• Cost per bag screened 


25 


3.2 




• Consumer satisfaction 


25 





Source: TSA. 



TSA has not yet established performance targets for the various 
components of the screening indexes, including performance targets for 
covert testing (person probability of detection). TSA's strategic plan states 
that the agency will use the performance data it collects to make tactical 
decisions based on performance. The screening performance indexes 
developed by TSA can be a useful analysis tool, but without targets for 
each component of the index, TSA will have difficulty performing 
meaningful analyses of the parts that add up to the index. For example, 
without performance targets for covert testing, TSA will not have 
identified a desired level of performance related to screener detection of 
threat objects. Performance targets for covert testing would enable TSA to 
focus its improvement efforts on areas determined to be most critical, as 
100 percent detection capability may not be attainable. In January 2005, 
TSA officials stated that the agency plans to track the performance of 
individual index components and establish performance targets against 



58 TSA headquarters officials stated that their intent is to provide FSDs with various 
mechanisms to assess their screening effectiveness at the airports for which they are 
responsible. Specifically, they stated that FSDs are provided with TIP data and the results 
of OIAPR's covert testing at their airports. Additionally, they have access to the results of 
local covert testing at their airports. 
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which to measure these components. They further stated that they are 
currently collecting and analyzing baseline data to establish these targets 
and plan to finalize them by the end of fiscal year 2005. 



It has been over 2 years since TSA assumed responsibility for passenger 
and checked baggage screening operations at the nation's commercial 
airports. TSA has made significant accomplishments over this period in 
meeting congressional mandates related to establishing these screening 
operations. With the congressional mandates now largely met, TSA has 
turned its attention to assessing and enhancing the effectiveness of its 
passenger and checked baggage screening systems. An important tool in 
enhancing screener performance is ongoing training. As threats and 
technology change, the training and development of screeners to ensure 
they have the competencies — knowledge, skills, abilities, and behaviors — 
needed to successfully perform their screening functions become vital to 
strengthening aviation security. Without addressing the challenges to 
delivering ongoing training, including installing high-speed connectivity at 
airport training facilities, TSA may have difficulty maintaining a screening 
workforce that possesses the critical skills needed to perform at a desired 
level. In addition, without adequate internal controls designed to help 
ensure screeners receive required training that are also communicated 
throughout the agency, TSA cannot effectively provide reasonable 
assurances that screeners receive all required training. Given the 
importance of the Online Learning Center in both delivering training and 
serving as the means by which the completion of screener training is 
documented, TSA would benefit from having a clearly defined plan for 
prioritizing the deployment of high-speed Internet/intranet connectivity to 
all airport training facilities. Such a plan would help enable TSA to move 
forward quickly and effectively in deploying high-speed connectivity once 
funding is available. 

Additionally, history demonstrates that U.S. commercial aircraft have long 
been a target for terrorist attacks through the use of explosives carried in 
checked baggage, and covert testing conducted by TSA and DHS OIG have 
identified that weaknesses and vulnerabilities continue to exist in the 
passenger and checked baggage screening systems, including the ability of 
screeners to detect threat objects. While covert test results provide an 
indicator of screening performance, they cannot solely be used as a 
comprehensive measure of any airport's screening performance or any 
individual screener's performance, or in determining the overall 
performance of federal versus private-sector screening. Rather, these data 
should be considered in the larger context of additional performance data, 
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such as TIP and recertification test results, when assessing screener 
performance. While TSA has undertaken efforts to measures and 
strengthen performance, these efforts have primarily focused on 
passenger screening and not on checked baggage screening. TSA's plans 
for implementing TIP for checked baggage screening, and establishing an 
image recognition component for the checked baggage screeners 
recertification testing — plans made during the course of our review — 
represent significant steps forward in its efforts to strengthen checked 
baggage screening functions. Additionally, although TSA has developed 
passenger and checked baggage screening effectiveness measures, the 
agency has not yet established performance targets for the individual 
components of these measures. Until such targets are established, it will 
be difficult for TSA to draw more meaningful conclusions about its 
performance and how to most effectively direct its improvement efforts. 
For example, performance targets for covert testing would enable TSA to 
focus its improvement efforts on areas determined to be most critical, as 
100 percent detection capability may not be attainable. We are encouraged 
by TSA's recent plan to establish targets for the individual components of 
the performance indexes. This effort, along with the additional 
performance data TSA plans to collect on checked baggage screening 
operations, should assist TSA in measuring and enhancing screening 
performance and provide TSA with more complete information with which 
to prioritize and focus its screening improvement efforts. 



To help ensure that all screeners have timely and complete access to 
screener training available in the Online Learning Center and to help 
provide TSA management with reasonable assurance that all screeners are 
receiving required passenger and checked baggage screener training, we 
recommend that the Secretary of the Department of Homeland Security 
direct the Assistant Secretary, Transportation Security Administration, to 
take the following two actions: 

• develop a plan that prioritizes and schedules the deployment of high-speed 
Internet/intranet connectivity to all TSA's airport training facilities to help 
facilitate the delivery of screener training and the documentation of 
training completion, and 

• develop internal controls, such as specific directives, clearly defining 
responsibilities for monitoring and documenting the completion of 
required training, and clearly communicate these responsibilities 
throughout the agency. 



Recommendations for 
Executive Action 



Page 46 



GAO-05-457 Screener Training and Performance Measurement Strengthened 



Agency Comments 
and Our Evaluation 



We provided a draft of this report to DHS for review and comment. On 
February 4, 2005, we received written comments on the draft report, which 
are reproduced in full in appendix V. DHS generally concurred with the 
findings and recommendations in the report, and agreed that efforts to 
implement our recommendations are critical to successful passenger and 
checked baggage screening training and performance. With regard to our 
recommendation that TSA develop a plan that prioritizes and schedules 
the deployment of high-speed Internet/intranet connectivity to all TSA's 
airport training facilities, DHS stated that TSA has developed such a plan. 
However, although we requested a copy of the plan several times during 
our review and after receiving written comments from DHS, TSA did not 
provide us with a copy of the plan. Therefore, we cannot assess the extent 
to which the plan DHS referenced in its written comments fulfills our 
recommendation. In addition, regarding our recommendation that TSA 
develop internal controls clearly defining responsibilities for monitoring 
and documenting the completion of required training, and clearly 
communicate those responsibilities throughout TSA, DHS stated that it is 
taking steps to define responsibility for monitoring the completion of 
required training and to insert this accountability into the performance 
plans of all TSA supervisors. TSA's successful completion of these ongoing 
and planned activities should address the concerns we raised in this 
report. DHS has also provided technical comments on our draft report, 
which we incorporated where appropriate. 



As agreed with your office, we will send copies of this report to relevant 
congressional committees and subcommittees and to the Secretary of the 
Department of Homeland Security. We will also make copies available to 
others upon request. In addition, the report will be made available at no 
charge on GAO's Web site at http://www.gao.gov. 
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If you have any questions about this report or wish to discuss it further, 
please contact me at (202) 512-8777. Key contributors to this report are 
listed in appendix VI. 

Sincerely yours, 




Cathleen A. Berrick 
Director, Homeland Security 
and Justice Issues 
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Appendix I: Summary of Previous Findings 
Related to Screener Training and 
Performance 



Product date 



Summary of previous findings related to screener training and performance 



Title and GAO product 
number 



September 2003 



The Transportation Security Administration (TSA) had deployed a basic screener 
training program and required remedial training but had not fully developed or 
deployed a recurrent training program for screeners or supervisors. 

TSA had collected little information to measure screener performance in detecting 
threat objects. 

• TSA's Office of Internal Affairs and Program Review's (OIAPR) covert testing 
was the primary source of information collected on screeners' ability to detect 
threat objects. However, TSA did not consider the covert testing a measure of 
screener performance. 

• TSA was not using the Threat Image Projection system (TIP) but planned to 
fully activate the system with significantly more threat images than previously 
used in October 2003. 

• TSA had not yet implemented an annual proficiency review to ensure that 
screeners met all qualifications and standards required to perform their 
assigned screening functions. 



Airport Passenger 
Screening: Preliminary 
Observations Made and 
Challenges Remaining 

GAO-03-1173 



November 2003 Although little data existed on the effectiveness of passenger screening, TSA was 

implementing several efforts to collect performance data. 

• TSA's OIAPR had conducted little covert testing of the screener workforce but 
planned to double the number of tests it conducted during fiscal year 2004. 

• TSA only recently began activating TIP on a wide-scale basis and expected it 
to be fully operational at every checkpoint at all airports by April 2004. 

• TSA only recently began implementing the annual recertification program and 
did not expect to complete testing at all airports until March 2004. 

• TSA was developing performance indexes for individual screeners and the 
screening system as a whole but had not fully established these indexes. TSA 
expected to have them in place by the end of fiscal year 2004. 

November 2003 Although TSA had not fully developed or deployed recurrent or supervisory 

training programs, it was in the process of 

• deploying six recurrent training modules and was pilot-testing an online 
learning management system, and 

• working with the U.S. Department of Agriculture's Graduate School to tailor its 
off-the-shelf supervisory course to meet the specific training needs of screening 
supervisors. 

February 2004 While TSA had taken steps to enhance its screener training programs, staffing 

imbalances, and lack of high-speed connectivity at airport training facilities had 
made it difficult for screeners at some airports to fully utilize these programs. 

Although TSA was making progress in measuring the performance of passenger 
screeners, it had collected limited performance data related to its checked 
baggage screening operations. However, TSA had begun collecting additional 
performance data related to its checked baggage screening operations and 
planned to increase these efforts in the future. 

As part of its efforts to develop performance indexes, TSA was developing 
baseline data for fiscal year 2004 and planned to report the indexes to DHS in 
fiscal year 2005. 



Aviation Security: Efforts to 
Measure Effectiveness and 
Address Challenges 

GAO-04-232T 



Aviation Security: Efforts to 
Measure Effectiveness and 
Strengthen Security 
Programs 

GAO-04-285T 



Aviation Security: 
Challenges Exist in 
Stabilizing and Enhancing 
Passenger and Baggage 
Screening Operations 

GAO-04-440T 
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Appendix I: Summary of Previous Findings 
Related to Screener Training and 
Performance 



Product date 



Summary of previous findings related to screener training and performance 



Title and GAO product 
number 



April 2004 



With the exception of covert testing and recent TIP data, data were not yet 
available to assess how well screeners were performing and what steps if any 
TSA needed to take to improve performance. Also, TSA was not using TIP as a 
formal indicator of screening performance, but instead was using it to identify 
individual screener training needs. 



Aviation Security: Private 
Screening Contractors 
Have Little Flexibility to 
Implement Innovative 
Approaches 

GAO-04-505T 



Source: GAO. 
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Appendix II: Objectives, Scope, and 
Methodology 



To examine efforts by the Transportation Security Administration to 
enhance their passenger and checked baggage screening programs, we 
addressed the following questions: (1) What actions has TSA taken to 
enhance training for screeners and supervisors? (2) How does TSA 
monitor compliance with screener training requirements? (3) What is the 
status of TSA's efforts to assess and enhance screener performance in 
detecting threat objects? 

To determine how TSA has enhanced training for screeners and 
supervisors and how TSA has monitored compliance with screener 
training requirements, we obtained and analyzed relevant legislation, as 
well as TSA's training plans, guidance, and curriculum. We reviewed data 
from TSA's Online Learning Center and assessed the reliability of the 
Online Learning Center database. We compared TSA's procedures for 
ensuring that screeners receive required training according to Standards 
for Internal Controls in the Federal Government. We interviewed TSA 
officials from the Office of Workforce Performance and Training and the 
Office of Aviation Operations in Arlington, Virginia. At the airports we 
visited, we interviewed Federal Security Directors and their staffs, such as 
Training Coordinators. We also met with officials from four aviation 
associations — the American Association of Airport Executives, Airports 
Council International, the Air Transport Association, and the Regional 
Airline Association. We did not assess the methods used to develop TSA's 
screener training program, nor did we analyze the contents of TSA's 
curriculum. Although we could not independently verify the reliability of 
all of this information, we compared the information with other supporting 
documents, when available, to determine data consistency and 
reasonableness. We found the data to be sufficiently reliable for our 
purposes. 

To determine what efforts TSA has taken to assess and to enhance 
screener performance in detecting threat objects, we reviewed related 
reports from the Department of Transportation and the Department of 
Homeland Security (DHS) Inspector General, Congressional Research 
Service, and TSA, as well as prior GAO reports. We obtained and reviewed 
TSA's covert test data and results of the annual recertification testing. 
(Results of the covert testing are classified and will be the subject of a 
separate classified GAO report.) We discussed methods for inputting, 
compiling, and maintaining the data with TSA officials. We also assessed 
the methodology of TSA's covert tests and questioned OIAPR officials 
about the procedures used to ensure the reliability of the covert test data. 
When we found discrepancies between the data OIAPR maintained in 
spreadsheets and the data included in the hard copy reports we obtained 
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Appendix II: Objectives, Scope, and 
Methodology 



from TSA, we worked with OIAPR to resolve the discrepancies. Further, 
we visited TSA headquarters to review TSA's annual recertification testing 
modules and discuss TSA's process for validating the recertification 
exams. As a result, we determined that the data provided by TSA were 
sufficiently reliable for the purposes of our review. We also reviewed 
TSA's performance measures, targets, and indexes. Finally, we interviewed 
TSA headquarters officials from several offices in Arlington, Virginia, 
including Aviation Operations, Workforce Performance and Training, 
Strategic Management and Analysis, and Internal Affairs and Program 
Review. 

In addition, in accomplishing our objectives, we also conducted site visits 
at select airports nationwide to interview Federal Security Directors and 
their staffs and conducted two Web-based surveys of Federal Security 
Directors. Specifically, we conducted site visits at 29 airports (13 category 
X airports, 9 category I airports, 3 category II airports, 3 category III 
airports, and 1 category IV airport) to observe airport security screening 
procedures and discuss issues related to the screening process with TSA, 
airport, and airline officials. We chose these airports to obtain a cross- 
section of all airports by size and geographic distribution. In addition, we 
selected each of the five contract screening pilot airports. The results from 
our airport visits provide examples of screening operations and issues but 
cannot be generalized beyond the airports visited because we did not use 
statistical sampling techniques in selecting the airports. The category X 
airports we visited were Baltimore Washington International Airport, 
Boston Logan International Airport, Chicago O'Hare International Airport, 
Dallas/Fort Worth International Airport, Denver International Airport, 
Washington Dulles International Airport, John F. Kennedy International 
Airport, Los Angeles International Airport, Newark Liberty International 
Airport, Orlando International Airport, Ronald Reagan Washington 
National Airport, San Francisco International Airport, Seattle-Tacoma 
International Airport. The category I airports we visited were Burbank- 
Glendale-Pasadena Airport, John Wayne Airport, Chicago Midway 
International Airport, Dallas Love Field, Kansas City International Airport, 
Little Rock National Airport, Metropolitan Oakland International Airport, 
Portland International Airport, and Tampa International Airport. The 
category II airports we visited were Jackson International Airport, Dane 
County Regional Airport, and Greater Rochester International Airport. The 
category III airports we visited were Idaho Falls Regional Airport, Jackson 
Hole Airport, and Orlando Sanford International Airport. The category IV 
airport we visited was Tupelo Regional Airport. 
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Further, we administered two Web-based surveys to all 155 Federal 
Security Directors who oversee security at each of the airports falling 
under TSA's jurisdiction. One survey, the general survey, contained 
questions covering local and national efforts to train screeners and 
supervisors and the status of TSA's efforts to evaluate screener 
performance, including the annual recertification program and TIP. The 
second survey attempted to gather more specific airport security 
information on an airport(s) under the Federal Security Director's 
supervision. For the airport-specific survey, each Federal Security 
Director received one or two surveys to complete, depending on the 
number of airports they were responsible for. Where a Federal Security 
Director was responsible for more than two airports, we selected the first 
airport based on the Federal Security Director's location and the second 
airport to obtain a cross-section of all airports by size and geographic 
distribution. In all, we requested information on 265 airports. However, 
two airports were dropped from our initial selection because the airlines 
serving these airports suspended operations and TSA employees were 
redeployed to other airports. As a result our sample size was reduced to 
263 airports, which included all 21 category X, and 60, 49, 73, and 
60 category I through IV airports respectively. In that we did not use 
probability sampling methods to select the sample of airports that were 
included in our airport-specific survey, we cannot generalize our findings 
beyond the selected airports. 

A GAO survey specialist designed the surveys in combination with other 
GAO staff knowledgeable about airport security issues. We conducted 
pretest interviews with six Federal Security Directors to ensure that the 
questions were clear, concise, and comprehensive. In addition, TSA 
managers and an independent GAO survey specialist reviewed the survey. 

We conducted these Web-based surveys from late March to mid-May 2004. 
We received completed general surveys from all 155 Federal Security 
Directors and completed airport-specific surveys for all 263 separate 
airports for which we sought information, for 100 percent response rates. 
The surveys' results are not subject to sampling errors because all Federal 
Security Directors were asked to participate in the surveys and we did not 
use probability-sampling techniques to select specific airports. However, 
the practical difficulties of conducting any survey may introduce other 
errors, commonly referred to as nonsampling errors. For example, 
difficulties in how a particular question is interpreted, in the sources of 
information that are available to respondents, or in how the data are 
entered into a database or were analyzed can introduce unwanted 
variability into the survey results. We took steps in the development of the 
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Methodology 



surveys, the data collection, and the data editing and analysis to minimize 
these nonsampling errors. Also, in that these were Web-based surveys 
whereby respondents entered their responses directly into our database, 
there was little possibility of data entry or transcription error. In addition, 
all computer programs used to analyze the data were peer reviewed and 
verified to ensure that the syntax was written and executed correctly. 

We performed our work from May 2003 through April 2005 in accordance 
with generally accepted government auditing standards. Certain 
information we obtained and analyzed regarding screener training and 
performance are classified or are considered by TSA to be sensitive 
security information. Accordingly, the results of our review of this 
information are not included in this report. 1 



*We issued two additional reports detailing the results of our review, which discuss 
information deemed to be classified or sensitive security information. The report 
containing sensitive security information is GAO, Aviation Security: Screener Training 
and Performance Measurement Strengthened, but More Work Remains, GAO-05-143SU 
(Washington D.C.: Feb. 28, 2005). The report containing classified and sensitive security 
information is GAO, Results of Transportation Security Administration's Covert Testing 
for Passenger and Checked Baggage Screening for September 2002 through 
September 2004, GAO-05-437C (Washington D.C.: Apr. 7, 2005). 
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Appendix III: TSA Screener Training Tools 
Designed to Help Improve Screener 
Performance 



i raining 1001 
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Hand Held Metal Detector 
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Provide an informative and effective learning tool to enhance 
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searches of passengers. 


Deployed November 21, 2003 


MBS II Weapons Kits 


This tool allows screeners to touch actual improvised explosive 
device (IED) components and build their own devices. This 

cXpcllcl Uld.1 IccHlllliy Will UlldUlU oOIggHgio IU lllUlfc? IcaUlly UclcUl leal 

lEDs during screening. These weapons are also used to assist in 
training by using them for live testing conducted by FSD staff. 


Deployed January 26, 2004 


Firearms Weapons Kits 


This tool allows screeners to touch actual firearms and begin to 
understand how they can be broken down into various parts. By 
understanding this and experiencing it, screeners are better able to 
see the components of a firearm during actual screening. These 
weapons are also used to assist in training by using them for live 
testing conducted by FSD staff. 


Deployed January 26, 2004 


X-Ray Operator Video 


Maintain and enhance the screeners' X-ray image operational skills. 


Deployed February 5, 2004 


X-Ray Tutor Version 1 


Provide a tool that includes about 14,000 image combinations to 
practice threat identification. 


Deployed February 15, 2004 


Basic Supervisory Technical 
Web-Based Training 1 


Assist screener supervisors to understand additional roles they must 
perform, including enhanced technical skills needed to adequately 
supervise screening functions and resolve alarms using interactive, 
performance-based training tool 


Deployed April 16, 2004 


Mobile Training Assist 
Teams (MTAT) 


These teams go into airports where data shows performance needs 
attention. The team offers a variety of services to assist in improving 
the performance, such as on-the-spot training and consulting 
services. Team visits can be initiated by FSDs, Internal Affairs 
reports, Quality Assurance trips, or MTAT Supervisors proactively 
visiting the airport and FSD. 


Site visits completed from 
October 2003 through 
December 3, 2004: 

• North Central (37 visits) 

• South Central (51 visits) 

• Northeast (25 visits) 

• Southeast (60 visits) 

• Western (53 visits) 

• 54 FSD Stakeholder 
Relations Meetings 


Basic Screener Supervisory 
Classroom Training 


Improve screener supervisors' knowledge of federal government and 
TSA personnel rules and how to effectively coach and communicate 
with employees. 


Approximately 3,800 supervisors 
have been trained. 


ETD Maintenance Course 


Certification of screeners to perform supervisory maintenance tasks 
above and beyond operator training. 


Delivered April 1 , 2004 


Credential Verification 
Training 


Provide students with basic skills needed to verify the identity of flying 
armed law enforcement officers. 


Deployed April 15, 2004 


Threat in the Spotlight 


This weekly product brings to light actual cases of weapons being 
found by law enforcement, with an explanation of how those weapons 
could be used to attack aviation. 


Deployed weekly in 2004 


EDS Operations Web- 
Based Training 


Provide interactive, performance based recurrent Web-based training 
modules for checked baggage explosive detection systems (EDS). 


Deployed April 30, 2004 
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Appendix III: TSA Screener Training Tools 
Designed to Help Improve Screener 
Performance 



Training tool 


Purpose 


Status reported by TSA 
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i idi iuvv&i iuii ly cii iu r di 

Down Web-Based Training 
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complementary to Hand Held Metal Detector and Pat Down Video 
that allows the screener to practice proper techniques and receive 
immediate feedback. 


Rpnlnx/prl Anril ^0 OC)C)d 


Customer Service Web- 
Based Training 


Reinforces TSA's customer service principles and places the 
screener in various situations requiring effective customer service 
responses. 


Deployed April 30, 2004 


Checkpoint and Checked 
Baggage Operations Web- 
Based Training 


Provide interactive, performance-based recurrent training modules for 
checkpoint and checked baggage operations. 


Deployed April 30, 2004 


Physical Bag Search Video 


Maintain and enhance screeners' explosive trace detection (ETD) 
and physical bag search skills for carry-on and checked baggage. 


Deployed April 30, 2004 


ETD and Physical Bag 
Search Web-Based Training 


Provide interactive recurrent Web-based training modules for ETD 
and physical bag search. 


Deployed April 30, 2004 


Based Training 


Pm\/iHo an intorar w ti\/o norfr\rnnanpo.hacoH traininn tool to onhanro 
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screener's ability to identify prohibited items. 




Effectively Screening 
Prosthetics Video 


Provide an informative and effective learning tool to maintain and 
enhance the skills of screeners in the areas of persons with 
prosthetics. 


Deployed December 16, 2004 


X-Ray Tutor Version 2 


Provide a tool to practice threat identification with about 10,000,000 
image combinations. 


Scheduled to be deployed 
during the second quarter of 
fiscal year 2005 


X-Ray Simulator Training 


Sharing the X-Ray Tutor Version 2 library, this tool will allow 
screeners to practice finding threat items using the full capabilities of 
the TIP-ready X-ray machines. 


Scheduled to be deployed 
during the second quarter of 
fiscal year 2005 


Basic Supervisory Technical 
Web-Based Training 2 


Provide an interactive, performance-based tool to convey how the 
supervisor is to handle screening situations, handed off by the 
screening, following standard operator procedures. 


To be developed 


Safety, Lifting, and Twisting 
Web-Based Training 


Provide a Web-based training that will engage the student with 3- 
dimensional representations of the muscular frame, showing proper 
lifting techniques and the results of improper techniques. 


Scheduled to be deployed 
during the second quarter of 
fiscal year 2005 



Source: TSA. 
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Appendix IV: Summary of TSA's Short-Term 
Action Items for Strengthening Passenger 
Screener Performance 



People Action item 



Description 



Benefit 



1 



Increase FSD support 
and accountability 



Hold FSDs accountable for screening 
performance and delivery of security 



Management accountability is driven down to the local 
airport 

FSD performance is linked to screener performance, 
creating incentives for maintaining and improving 
security 



Enhance training 



Provide ongoing training for screeners 
and supervisors to maintain their skills 
and provide new skills and techniques 
based on evolving threats and lessons 
learned 



Maintains and improves knowledge base of screeners 

Ensures proper oversight by supervisors 

Ensures that screeners are capable of addressing 
evolving threats 



Increase Internal 
Affairs covert testing 



Increase the frequency of TSA covert 
testing 



Improved identification of systemic vulnerabilities in 
airport security systems 

Immediate implementation of limited remedial actions 



Continue to pursue 
human performance 
improvements 



Better understand reasons and causes for 
human errors and interactions with 
technology in order to identify 
opportunities for performance 
improvements, with a goal of identifying 
optimum work conditions 



Reduces human-based errors 

Increases workforce morale and working conditions, 
leading to improved performance 



Technology 


5 


Continue to identify 
screening technology 
improvements 


Continue to research alternative 
technologies and seek short-term 
technological solutions, especially for 
potential vectors 


Identifies threats more accurately and quickly 
Decreases number of false positives from equipment 


6 


Finish installing TIP 


The TIP system is a series of 2,400 
images of threat objects that can be 
automatically fed into X-ray machines 
during actual screening 


Maintains alertness of screeners 

Identifies individual screener performance issues 


7 


Expedite high-speed 
connectivity to 
checkpoints and 
training computers 


Connect all TSA offices, checkpoints and 
screening equipment (X-rays, EDS 
machines) to the Internet in order to 
automate and improve processes that are 
currently done manually or not at all 


Provides immediate feedback on and response to 
screener performance issues 

Improves communication with managers in the field 


Process 


8 


Refresh aviation 
operations policy, 
procedures, and 
practice 


Conduct a thorough and expedited review 
of all policies and procedures developed 
during the rollout of TSA with a focus on 
increasing screening performance and 
capabilities 


Maintains "freshness" of standard operating 
procedures based on most recent intelligence about 
security threats 

Removes or updates outdated or unnecessary 
screening techniques based on lessons learned 


9 


Improve workforce 
management 


Determine the optimal workforce staffing 
levels based on latest passenger flows 
and other factors 


Maximizes utilization of existing resources 
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Appendix V: Comments from the Department 
of Homeland Security 



U.S. Deparhnenl of Homeland Security 

Washington, DC 20528 




Homeland 
Security 



February 4, 2005 



Ms. Cathleen Berrick 

Director, Homeland Seeurity & Justice Issues 
U.S. Government Accountability Office 
441 G Street, N.W. 
Washington, DC. 20548 

Dear Ms. Berrick: 

RE: GAO-05-143SU, Aviation Security: Screener Training and Performance 
Measurement Strengthened, but More Work Remains 
(GAO Job Code 440256) 

Thank you for the opportunity to comment on the subject draft report. The Department 
of Homeland Security (DHS) appreciates the work done to identify areas for 
improvement in the Transportation Security Administration's training and performance 
measurement of passenger and baggage screeners. We generally concur with the report 
and its recommendations and appreciate the discussion of challenges, and related on- 
going and planned work designed to fully meet our responsibilities. 

The report acknowledges the substantial progress the Transportation Security 
Administration (TSA) has made in enhancing screener training, and measuring and 
enhancing performance. However, there are areas within the report about which DHS 
would like to comment. 

The U.S. Government Accountability Office (GAO) made the following two 
recommendations regarding passenger and baggage screening training and performance: 

1. Develop a plan that prioritizes and schedules the deployment of high-speed 
internet/intranet connectivity to all TSA's airport training facilities to help 
facilitate the delivery of screener training and the documentation of training 
completion. 

2. Develop internal controls, such as specific directives, clearly defining 
responsibilities for monitoring and documenting the completion of required 
training, and clearly communicate these responsibilities throughout [TSA]. 

DHS agrees that efforts to implement these recommendations are critical to successful 
passenger and baggage screening training and performance. We believe that we have 
fulfilled the first recommendation because TSA has developed such a plan. 



www.dhs.gov 
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Appendix V: Comments from the Department of Homeland 
Security 



2 



Specifically, TSA has developed a plan to facilitate connectivity to all of TSA's airport 
training facilities. The High-Speed Operational Connectivity (HiSOC) program is a 
detailed plan and corresponding schedule for ensuring that training centers in airports 
receive high speed connectivity. The $174 million of funding necessary to complete 
HiSOC installation is included in the President's FY 2006 budget request. To date in 
FY 2005, $3 million has been allocated to the HiSOC effort. As additional funds are 
made available for the HiSOC program, connectivity can be deployed to additional 
training centers. 

The HiSOC program includes a detailed plan for Wide Area Network (WAN) 
connectivity to TSA Airports including the following services: 

• Local Area Networking (LAN) linking Operation Centers, Training Centers and 
Break Rooms, Checkpoint/Passenger Screening Areas and Baggage Screening 
Areas, and Federal Security Directors 

• XP Migration 

• Intelligent Phone Deployment 
Email Migration 

• Remote Access via Virtual Private Network (VPN) 

TSA is already taking steps to address the second recommendation. The TSA Executive 
Leadership is drafting a memorandum to advise all Assistant Administrators and Federal 
Security Directors that managers and supervisors will be held accountable for their 
subordinates completing all mandatory training requirements. This accountability will be 
inserted into the performance plans of all TSA supervisors. 

In February 2005, TSA's Office of Workforce Performance and Training will be 
updating the existing Management Directive on Training Records, This directive will be 
revised to strengthen and clarity recordkeeping requirements. By the end of March 2005, 
the TSA Online Learning Center (OLC) should be enhanced to include a robust reporting 
tool that will produce a number of detailed and summary level training performance 
accountability reports directly accessible to local training administrators. By the summer 
2005, management plans to expand TSA's Performance Information Management 
System to include select OLC training summary data. This data will be visible to 
managers and will include the ability to correlate training performance data with other 
TSA source data for cause and effect and trending analyses. 



Sincerely, 



Steven Pecinovsky u 
Acting Director 

Departmental GAO/OIG Liaison Office 




MMcP 
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GAO's Mission 



Obtaining Copies of 
GAO Reports and 
Testimony 



The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting its 
constitutional responsibilities and to help improve the performance and 
accountability of the federal government for the American people. GAO 
examines the use of public funds; evaluates federal programs and policies; 
and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO's 
commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. To 
have GAO e-mail you a list of newly posted products every afternoon, go 
to www.gao.gov and select "Subscribe to Updates." 



Order by Mail or Phone 



The first copy of each printed report is free. Additional copies are $2 each. 
A check or money order should be made out to the Superintendent of 
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. Orders 
should be sent to: 

U.S. Government Accountability Office 
441 G Street NW, Room LM 
Washington, D.C. 20548 



To order by Phone: 



Voice: (202) 512-6000 
TDD: (202) 512-2537 
Fax: (202) 512-6061 



To Report Fraud, 
Waste, and Abuse in 
Federal Programs 

Congressional 
Relations 

Public Affairs 



Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 
E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 
U.S. Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonPl@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548 
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